Visual processing device, display device, visual processing method, program, and integrated circuit

ABSTRACT

A visual processing device, display device, visual processing method, program, and integrated circuit that change a strength of visual processing of an image in real-time. A spatial processing portion ( 2 ) creates an unsharp signal US from an input signal IS. A target level setting portion ( 4 ) sets a predetermined target level value L for setting a range according to which the strength of the visual processing is adjusted. An effect adjustment portion ( 5 ) creates a synthesized signal MUS by synthesizing the predetermined target level value L and the unsharp signal US in accordance with an effect adjustment signal MOD that has been set externally. A visual processing portion ( 3 ) outputs an output signal OS in accordance with the input signal IS and the synthesized signal MUS, making it possible to change the strength of the visual processing.

BACKGROUND OF THE INVENTION

1. Field of Invention

The present invention relates to visual processing devices, displaydevices, image-capturing devices, portable information devices, andintegrated circuits, and in particular relates to visual processingdevices, display devices, image-capturing devices, portable informationdevices, and integrated circuits in which the strength of visualprocessing of an image is changed.

2. Description of the Related Art

Spatial processing and tone processing (tone mapping processing) areknown as methods for visually processing an image signal of an originalimage.

In spatial processing, a target pixel to be filtered is processed usingthe pixels surrounding the target pixel. Another method that is known isto use an image signal that has been spatially processed in order toperform contrast enhancement or dynamic range (hereinafter, abbreviatedas “DR”) compression, for example, of the original image (for example,see Patent Document 1).

Tone processing is processing in which a lookup table (hereinafter,abbreviated as “LUT”) is used to effect pixel value conversion for eachtarget pixel, regardless of the pixels surrounding that target pixel,and is also known as gamma correction. For example, to enhance thecontrast, pixel value conversion is performed using a LUT for assigninga wide range of tone to frequently appearing tone levels (that cover alarge area) in the original image. Some examples of tone processing inwhich a LUT is employed include tone processing in which a single LUT isselected and used for an entire original image (histogram equalization),and tone processing in which a LUT is selected and used for each of aplural number of image regions obtained by partitioning the originalimage (local histogram equalization).

Conventional visual processing devices have been provided with aplurality of profile data with different conversion characteristics, andachieved the different visual processing modes discussed above byswitching the profile data (for example, see Patent Document 2).

FIG. 47 illustrates a conventional visual processing device 900. In FIG.47, the visual processing device 900 is made of a spatial processingportion 901 that executes spatial processing on the luminance value ofeach pixel of an original image that has been obtained as an inputsignal IS and outputs an unsharp signal US, and a visual processingportion 902 that uses the input signal IS and the unsharp signal US forthe same pixel to perform visual processing of the original image, andoutputs an output signal OS. The unsharp signal US is the brightnesssignal of a local region in which the luminance signal has beenprocessed by a low-pass filter, and is a blur signal. The visualprocessing portion 902 is constituted by a two-dimensional LUT.

The visual processing portion 902 executes gamma correction with thetone conversion characteristics shown in FIG. 48, and selects a toneconversion curve that corresponds to the unsharp signal US of a targetregion in the image in order to increase or decrease the contrast. Forexample, it selects the curve of unsharp signal USO to brighten darkregions in the image, whereas it selects the unsharp signal USn curve toinhibit brightness in bright regions and strengthen the contrast. Thisgroup of curves is called a profile.

A profile data registration device 903 is provided with profile groupsfor different types of visual processing, and in the visual processingportion 902 registers the most appropriate profile for a target visualprocessing.

The profile data registration device 903 also updates to requiredprofile data according to the strength of the visual processing.

For example, when it was desirable to change the strength of contrastenhancement in dark regions between when a face is extremely dark andwhen it is slightly dark even in a backlit image, then the brightnesswas adjusted by updating to profile data that have the most appropriatetone conversion characteristics.

Patent Document 1: U.S. Pat. No. 4,667,304.

Patent Document 2: International Disclosure Pamphlet No. 2005/027043.

BRIEF SUMMARY OF THE INVENTION Problem to be Solved by the Invention

However, with this conventional configuration, there was the problemthat because it is necessary to prepare profile data according to thestrength of the effect of visual processing, the amount of data becomeslarge. A large amount of data requires the memory capacity for storingthe profile data to also be large (normally about several 100 bytes toseveral 10 K bytes).

There also was the issue that it is necessary to update the profile databased on the strength of the effect of visual processing. Because alarge update time is required to update the profile data, there was theissue that the effect of visual processing could not be changed in realtime for each local region in the image. In particular, if the imageprocessing portion is constituted by a two-dimensional LUT, then thedata amount becomes large and an even greater update time is required.

One example of processing, for improving the image quality, whichresembles human vision, is visual processing in which the value of atarget pixel is converted based on the contrast between the value of thetarget pixel and the values of the pixels in the surrounding region. Insuch visual processing, to further enhance the processing effect, thebrightness information is extracted from a wide region around theposition of the target pixel.

However, since the value of the target pixel is determined from thecontrast between the value of the target pixel and the values of thesurrounding pixels, if there is a steep edge region in the surroundingregion, then the impact of the values of the surrounding pixels resultsin a visual processing output that changes gently near the edge, even inflat regions in which pixel values fluctuate little. When a large changein luminance occurs in a flat region, a shadow-like border occurs in theregion adjacent to the edge and results in an unnatural image. Also,when visual processing is performed on, for example, images with fewedge regions, images with few tone levels (luminance levels), images inwhich there is little difference in the luminance between adjacentpixels and many continuous analogous values occur, and block images thathave been partitioned into a plurality of blocks in which few blocksinclude high-frequency components (hereinafter, such images will bereferred to as “special images”) in the same manner as on naturalimages, the change in luminance of the flat regions easily stands outand shadow-like borders occur in regions adjacent to edges, producing anunnatural image.

The issue to be solved by the invention is to achieve a visualprocessing device, a display device, a visual processing method, aprogram, and an integrated circuit that are capable of suppressingartifacts even if an image with sharp edge regions or a special imagehas been input, and that with a simple configuration can change thestrength of the visual processing of the image in real time.

Means for Solving Problem

A 1st aspect of the invention is a visual processing device thatvisually processes and outputs an image signal that has been input, andthat is provided with an effect adjustment portion that performsprocessing for setting an effect of visual processing according to aneffect adjustment signal, and a visual processing portion that performsvisual processing on an image signal.

Thus, with a simple configuration it is possible to set the effect ofvisual processing with an effect adjustment signal. Consequently, theeffect of visual processing can be adjusted in real time.

A 2nd aspect of the invention is the 1st aspect of the invention,further including a target level setting portion that sets apredetermined target level, and a spatial processing portion thatperforms predetermined spatial processing on an image signal and outputsa processed signal. The effect adjustment portion outputs a synthesizedsignal that is obtained from synthesizing the predetermined target leveland the processed signal according to the effect adjustment signal forsetting the effect of visual processing. The visual processing portionconverts an image signal based on the synthesized signal and the imagesignal.

With this configuration, the strength of the visual processing ischanged by creating different synthesized signals with the effectadjustment signal, and thus it is not necessary to change apredetermined tone conversion function to match the degree of thestrength. If the predetermined tone conversion function is achieved by ahardware circuit, then it is possible to reduce the circuit scalebecause it is not necessary to have a plurality of circuits thatcorrespond to the strength of the visual processing. If thepredetermined tone conversion characteristics are achieved by profiledata stored in a 2D LUT, then it is possible to reduce the memorycapacity because it is not necessary to have a plurality of profile datacorresponding to the strength of the visual processing. Because it is nolonger necessary to update the profile data according to the strength ofthe visual processing, it is possible to change the strength of visualprocessing even when the visual processing portion is constituted by a2D LUT. By setting a target level with the target level setting portion,it is possible to set the tone conversion characteristics to which tochange the visual processing effect by the visual processing that isachieved by the visual processing device.

A 3rd aspect of the invention is the 1st aspect of the invention,further including a target level setting portion that sets apredetermined target level, a spatial processing portion that performspredetermined spatial processing on an image signal and outputs aprocessed signal, and a correction portion that corrects an imagesignal. The effect adjustment portion outputs a synthesized signalobtained by synthesizing the predetermined target level and theprocessed signal according to the effect adjustment signal for settingthe effect of visual processing. The visual processing portion outputs again signal based on the synthesized signal and the image signal. Thecorrection portion corrects an image signal based on the gain signal.

With this configuration, the strength of the visual processing can bechanged by creating different synthesized signals with the effectadjustment signal, and thus a predetermined gain function can be fixed.If the predetermined gain function is achieved by a hardware circuit,then it is possible to reduce the circuit scale because it is notnecessary to have a plurality of circuits that correspond to thestrength of the visual processing. If the predetermined gain function isachieved by profile data stored in a 2D LUT, then it is possible toreduce the memory capacity because it is not necessary to have aplurality of profile data that correspond to the strength of the visualprocessing. Also, because it is not necessary to update the profile datain correspondence with the strength of the visual processing, it ispossible to change the strength of the visual processing in real timeeven if the visual processing portion is constituted by a 2D LUT.

A 4th aspect of the invention is the 1st aspect of the invention,further including a spatial processing portion that performspredetermined spatial processing on the image signal and outputs aprocessed signal. The effect adjustment portion outputs a synthesizedsignal that is obtained by synthesizing the image signal and theprocessed signal according to the effect adjustment signal for settingthe effect of visual processing. The visual processing portion convertsthe image signal based on the synthesized signal and the image signal.

With this configuration, the effect adjustment portion creates asynthesized signal by interpolating the image signal and the processedsignal with the effect adjustment signal. Thus, it is possible to changethe effect of visual processing from the characteristics of only gammaconversion for converting a predetermined brightness, to thecharacteristics of converting the local contrast. It is also possible tofix a predetermined tone conversion function even though the effect ofvisual processing is to be changed.

A 5th aspect of the invention is the 1st aspect of the invention,further including a spatial processing portion that performspredetermined spatial processing on the image signal and outputs aprocessed signal, and a correction portion that corrects an imagesignal. The effect adjustment portion outputs a synthesized signal thatis obtained by synthesizing the image signal and the processed signalaccording to the effect adjustment signal for setting the effect ofvisual processing. The visual processing portion outputs a gain signalbased on the synthesized signal that has been synthesized and the imagesignal. The correction portion corrects an image signal based on thegain signal.

With this configuration, the effect adjustment portion creates asynthesized signal by interpolating the image signal and the processedsignal with the effect adjustment signal. Thus, it is possible to changethe effect of visual processing from the characteristics of only gammaconversion for converting a predetermined brightness, to thecharacteristics of converting the local contrast. It is also possible tofix a predetermined gain function even if the effect of visualprocessing is to be changed.

A 6th aspect of the invention is any one of the 1st through 5th aspectsof the invention, in which the visual processing portion includes atwo-dimensional lookup table.

With this configuration, it is possible to register profiles fordifferent visual effects, such as DR compression processing, localcontrast processing, and tone processing. Further, by storing data basedon the gain characteristics in a 2D LUT, it is possible to reduce thememory capacity more than if the gain conversion values are stored asdata as they are.

A 7th aspect of the invention is the 1st aspect of the invention,further including a surrounding image information extraction portionthat extracts surrounding image information of the image signal that hasbeen input, and an effect adjustment signal generation portion thatoutputs an effect adjustment signal for setting the effect of the visualprocessing. The visual processing portion visually processes an imagesignal based on the image signal and the surrounding image information.The effect adjustment portion sets the effect of the visual processingaccording to the effect adjustment signal.

With this configuration, it becomes possible to set (vary) the effect ofvisual processing according to the effect adjustment signal, and byadjusting the effect in regions where artifacts occur, it is possible tosuppress artifacts.

An 8th aspect of the invention is the 7th aspect of the invention,wherein the effect adjustment signal generation portion detects a regionthat is adjacent to an edge region from the image signal, and outputsthe effect adjustment signal.

Thus, it is also possible to inhibit artifacts near a edge region evenwhen an image that has a sharp edge region is input.

A 9th aspect of the invention is the 8th aspect of the invention,wherein the effect adjustment signal generation portion detects a flatregion that is adjacent to the edge region from the image signal, andoutputs the effect adjustment signal.

Thus, it is also possible to suppress artifacts in a flat region that isnear an edge region, in which artifacts stand out easily.

A 10th aspect of the invention is the 8th or 9th aspects of theinvention, wherein the effect adjustment signal generation portionoutputs the effect adjustment signal according to an amount of change ofthe surrounding image information.

Thus, it is also possible to inhibit artifacts that occur along withchanges in the surrounding image information.

An 11th aspect of the invention is the 8th or 9th aspects of theinvention, wherein the effect adjustment signal generation portionincludes a flatness detection portion that detects a degree of flatnessof the flat region whose difference in luminance with an adjacent regionis less than or equal to a predetermined value from the image signal,and an edge detection portion that detects an edge amount of an edgeregion whose difference in luminance with an adjacent region is lessthan or equal to a predetermined value from the image signal. The effectadjustment signal generation portion outputs the effect adjustmentsignal based on the outputs from the flatness detection portion and theedge detection portion.

Thus, additionally, even when an image that has sharp edge regions isinput, it is possible to suppress artifacts in flat regions that arenear edge regions.

A 12th aspect of the invention is any one of the 7th through 11thaspects of the invention, wherein the effect adjustment portion outputsa first synthesized signal that is obtained from synthesizing the imagesignal and the surrounding image information according to the effectadjustment signal. The visual processing portion visually processes theimage signal based on the first synthesized signal and the image signal.

Thus, additionally, it is possible for the visual processing portion toselect different tone conversion processing based on the firstsynthesized signal, and can visually process the image signal based onthe selected tone conversion processing, and thus can vary the effect ofvisual processing.

A 13th aspect of the invention is any one of the 7th through 11thaspects of the invention, wherein the effect adjustment portion outputsa second synthesized signal that is obtained by synthesizing the imagesignal and the output that has been visually processed by the visualprocessing portion according to the effect adjustment signal.

Thus, additionally, it is possible to perform the output changing theratio of the image signal and the processed signal in accordance withthe effect adjustment signal, and this allows the effect of visualprocessing to be differed.

A 14th aspect of the invention is the 1st aspect of the invention,further including a surrounding image information extraction portionthat extracts surrounding image information of the image signal that hasbeen input, and a special image detection portion that obtains a degreeindicating a degree of a special image by detecting statistical biasexisting in an image and outputs the degree as the effect adjustmentsignal. The visual processing portion outputs a processed signal that isobtained by visually processing an image signal based on the imagesignal and the surrounding image information. The effect adjustmentportion controls the effect of the visual processing in accordance withthe effect adjustment signal.

With this configuration, the visual processing effect can be maintainedeven if a normal image that is not a special image has been input, andartifacts can be inhibited if a special image has been input.

A 15th aspect of the invention is the 14th aspect of the invention,wherein the special image detection portion detects the statistical biasbased on a ratio of the number of regions in which the gradation isjudged to be in changes to the total number of regions in the image, ora ratio of the number of regions in which the gradation is judged to bein constant to the total number of regions in the image, in the imageformed by the image signal.

Thus, additionally, it is possible to detect a statistical bias from theproportion of regions in which the gradation changes, or from theproportion of regions in which the gradation does not change, in theimage of the image signal.

A 16th aspect of the invention is the 15th aspect of the invention,wherein the special image detection portion increases the degree towhich the ratio of regions in which the gradation is judged to be inchange to the total number of regions in the image is low, or when theratio of the number of regions in which the gradation is judged to be inconstant to the total number of regions in the image is high.

Thus, additionally it is possible to detect the degree to which an imageis a special image, and it is possible to output an effect adjustmentsignal that is suited for processing the special image.

A 17th aspect of the invention is the 16th aspect of the invention,wherein the special image detection portion detects the ratio of regionsin which the gradation is judged to be in change by detecting an edgecomponent in the regions of the image.

Thus, additionally it is possible to detect the ratio of regions inwhich the gradation changes from the edge component of the image.

An 18th aspect of the invention is the 16th aspect of the invention,wherein the special image detection portion detects the ratio of regionsin which the gradation does not change by detecting a degree of flatnessin the image.

Thus, it is also possible to detect the ratio of regions in which thegradation does not change from the degree of flatness in the image.

A 19th aspect of the invention is the 18th aspect of the invention,wherein the special image detection portion detects the degree offlatness based on the total of number of continuous pixels analogous toeach other in pixel value (tone level) or the number of tone levels.

Thus, additionally it is possible to detect the degree of flatness fromthe number of tone levels or the continuous length of analogous pixelsin the image.

A 20th aspect of the invention is the 17th aspect of the invention,wherein the special image detection portion has an edge detectionportion that detects edge amount for each pixel in an image formed bythe image signal, an edge density calculation portion that detects edgepixels whose edge amount is equal to or greater than a predeterminedvalue and calculates a ratio of the number of edge pixels to the totalnumber of pixels in the image signal, and a first effect adjustmentsignal generation portion that outputs the effect adjustment signalaccording to that ratio.

Thus, additionally it is possible to detect a special image from theedges in the image, and it is possible to create an effect adjustmentsignal that corresponds to the bias of the proportion of edge pixels inthe special image.

A 21st aspect of the invention is the 17th aspect of the invention,wherein the special image detection portion has a high-frequency blockdetection portion that detects a high-frequency block includinghigh-frequency component, from an image formed by the image signal,which has been partitioned into a plurality of blocks, a high-frequencyblock density detection portion that detects a ratio of the number ofthe high-frequency blocks to the number of the plurality of blocks, anda second effect adjustment signal generation portion that outputs theeffect adjustment signal according to the ratio.

Thus, additionally, it is possible to detect a special image bydetecting high-frequency blocks within the image, and it is possible tocreate an effect adjustment signal that corresponds to the bias of theproportion of high-frequency blocks in the special image.

A 22nd aspect of the invention is the 19th aspect of the invention,wherein the special image detection portion has a classifier (afrequency detection portion) that classifies the pixel in the imageformed by the image signal into a class based on a tone level of thepixel and counts the number of the pixels belonging to each class, afrequency determination portion (determination portion) that comparesthe number of pixels belonging to each class with a predeterminedthreshold value to detect a class with the number of pixels larger thanthe predetermined threshold value, a tone level number detection portionthat counts the number of classes detected by the determination portion,and a third effect adjustment signal generation portion that outputs theeffect adjustment signal according to the number of the counted classes.

Thus, additionally, it is possible to detect a special image from thenumber of tone levels in the image, and it is possible to create aneffect adjustment signal that corresponds to the bias in the number oftone levels in the special image.

A 23rd aspect of the invention is the 19th aspect of the invention,wherein the special image detection portion has an analogous pixeldetection portion, a continuous length detection portion, a meancontinuous length calculation portion, and a fourth effect adjustmentsignal generation portion. The analogous pixel detection portion detectsanalogous pixels whose difference in luminance with adjacent pixels isless than or equal to a predetermined value from the image signal. Thecontinuous length detection portion detects a continuous length in whichthe analogous pixels are continuous. The mean continuous lengthcalculation portion calculates a mean continuous length by averaging aplurality of the continuous lengths that have been detected by thecontinuous length detection portion. The fourth effect adjustment signalgeneration portion outputs the effect adjustment signal according to themean continuous length.

Thus, additionally, it is possible to detect a special image from themean continuous length of analogous pixels in the image, and it ispossible to create an effect adjustment signal that corresponds to thebias of the mean continuous length in the special image.

A 24th aspect of the invention is any one of the 14th through 23rdaspects of the invention, wherein the effect adjustment portion outputsa first synthesized signal that is synthesized by changing the ratio ofthe image signal and the surrounding image information according to theeffect adjustment signal, and wherein the visual processing portionvisually processes the image signal based on the first synthesizedsignal and the image signal.

Thus, additionally, it is possible for the visual processing portion toselect different tone conversion processing based on the firstsynthesized signal, so that it can differ the effect of the visualprocessing.

A 25th aspect of the invention is any one of the 14th through 23rdaspects of the invention, wherein the effect adjustment portion outputsa second synthesized signal that is synthesized by changing the ratio ofthe image signal and the processed signal according to the effectadjustment signal.

Thus, additionally, it is possible to perform the output changing theratio of the image signal and the processed signal according to theeffect adjustment signal, and this allows the visual processing effectto be differed.

A 26th aspect of the invention is any one of the 14th through 23rdaspects of the invention, wherein the visual processing portion includesa two-dimensional lookup table, and performs visual processing based oncharacteristic data that have been set in the two-dimensional lookuptable. The effect adjustment portion sets, in the visual processingportion, characteristic data that are synthesized by changing the ratioof a plurality of the characteristic data with different visualprocessing effects according to the effect adjustment signal.

Thus, additionally, it is possible to perform visual processing usingcharacteristic data that are synthesized by changing the ratio of aplurality of characteristic data with different visual processingeffects in accordance with the effect adjustment signal, and this allowsthe effect of visual processing to be differed.

A 27th aspect of the invention is any one of the 14th through 26thaspects of the invention, wherein the special image detection portioninputs a reduced image, in which the image signal has been reduced, anddetects special images, which have the statistical bias, from thereduced image and outputs the effect adjustment signal based on thestatistical bias.

Thus, additionally, the impact of noise when detecting a special imageis suppressed. It is also possible to reduce the number of computationsof the processing.

A 28th aspect of the invention is any one of the 14th through 27thaspects of the invention, wherein the special image detection portiondetects the statistical bias from a frame image one or more frame imagesprior when the image signal is a frame image, or from a field image oneor more field images prior when the image signal is a field image.

Thus, additionally, it is possible to detect a special image from theframe immediately prior, and it is possible to use an effect adjustmentsignal that corresponds to the bias of the information of the specialimage from the head of the frames. It is also possible to detect aspecial image from the field immediately prior, and it is possible touse an effect adjustment signal that corresponds to the bias of theinformation of the special image from the head of the fields.

A 29th aspect of the invention is the 28th aspect of the invention,further including a continuous changing portion for continuouslychanging the effect adjustment signal. The continuous changing portioncontinuously changes the effect adjustment signal between frames whenthe effect adjustment signal is output in frame units, or between fieldswhen the effect adjustment signal is output in field units.

Thus, additionally, it is possible to suppress sudden changes in theeffect adjustment signal between frames, and thereby suppress flickeringof the image between frames. It is also possible to suppress suddenchanges in the effect adjustment signal between fields, and therebysuppress flickering of the image between fields.

A 30th aspect of the invention is a display device that is provided witha data reception portion that receives image data that have beentransmitted or broadcast, a decoding portion that decodes the image datathat have been received into video data, the visual processing deviceaccording to any one of the first through 29th aspects of the inventionfor visually processing the decoded video data and outputting an outputsignal, and a display portion that performs a display of the outputsignal that has been visually processed by the visual processing device.

With this configuration, it is possible to change the strength of visualprocessing in real time through brightness adjustment of the image anddisplay this with a display device. It should be noted that in additionto a display device, it is also possible to achieve an image-capturingdevice and a portable information terminal device that are provided withthe visual processing device.

The image-capturing device can have a configuration in which it isprovided with an image-capturing portion that performs an image captureof an image, and a visual processing device that receives the image thathas been captured by the image-capturing portion as an input signal andperforms visual processing.

With this configuration, with an image-capturing device as well it ispossible to obtain the same effect as the visual processing devices.

The portable information device can have a configuration in which it isprovided with a data reception portion that receives image data thathave been communicated or broadcast, a visual processing device thatvisually processes the image data that have been received and outputs anoutput signal, and display means for performing a display of the outputsignal that has been visually processed.

With this configuration, with a portable information device as well itis possible to obtain the same effect as the visual processing devices.

The portable information device can have a configuration in which it isprovided with an image-capturing portion that performs an image captureof an image, a visual processing device that receives the image that hasbeen captured by the image-capturing portion as an input signal andperforms visual processing and outputs an output signal, and a datatransmission portion for transmitting the output signal that has beenvisually processed.

With this configuration, with a portable information device it ispossible to obtain the same effect as the visual processing devices.

A 31st aspect of the invention is a visual processing method of visuallyprocessing and outputting an image signal that has been input, andincludes an effect adjustment step of performing processing for settingan effect of visual processing according to an effect adjustment signal,and a visual processing step of performing visual processing on an imagesignal.

Thus, the effect of visual processing can be easily set with an effectadjustment signal. Consequently, it is possible to adjust the effect ofvisual processing in real time.

A 32nd aspect of the invention is the 31st aspect of the invention,further including a target level setting step of setting a predeterminedtarget level, and a spatial processing step of performing predeterminedspatial processing on an image signal and outputting a processed signal.In the effect adjustment step, a synthesized signal obtained fromsynthesizing the predetermined target level and the processed signalaccording to the effect adjustment signal is output. In the visualprocessing step, tone conversion of an image signal is performed basedon the synthesized signal that has been synthesized and the imagesignal.

Thus, the strength of the visual processing is changed by creatingdifferent synthesized signals with the effect adjustment signal, andthus it is not necessary to change a predetermined tone conversionfunction to match the degree of the strength. Also, by setting a targetlevel in the target level setting step, it is possible to set the toneconversion characteristics to which to change the visual processingeffect that is achieved by the visual processing method.

A 33rd aspect of the invention is the 31st aspect of the invention,further including a surrounding image information extraction step ofextracting the surrounding image information of an image signal that hasbeen input, and an effect adjustment signal generation step ofoutputting an effect adjustment signal for setting an effect of thevisual processing. In the visual processing step, an image signal isvisually processed based on the image signal and the surrounding imageinformation. In the effect adjustment step, the effect of visualprocessing is set according to the effect adjustment signal.

With this method, it is possible to differ the visual processing effectaccording to the effect adjustment signal.

A 34th aspect of the invention is the 33rd aspect of the invention, inwhich, in the effect adjustment signal generation step, a region that isadjacent to an edge region is detected from the image signal, and theeffect adjustment signal is output.

Thus, additionally, even when an image that has a sharp edge region isinput, it is possible to suppress artifacts in a flat region that isnear the edge region.

A 35th aspect of the invention is the 31st aspect of the invention,further including a surrounding image information extraction step ofextracting the surrounding image information of an image signal that hasbeen input, and a special image detection step of obtaining a degreeindicating a degree of a special images by detecting statistical biasexisting in an image and outputting the degree as the effect adjustmentsignal. In the visual processing step, an image signal is visuallyprocessed based on the image signal and the surrounding imageinformation. In the effect adjustment step, the effect of the visualprocessing is set according to the effect adjustment signal.

With this method, the visual processing effect can be maintained even ina case where a normal image that is not a special image has been input,and artifacts can be inhibited in a case where a special image has beeninput.

A 36th aspect of the invention is a program for causing a computer toexecute an effect adjustment step outputting a control signal forsetting an effect of visual processing according to an effect adjustmentsignal, and a visual processing step of performing visual processing onan image signal based on the control signal and the image signal, inorder to perform visual processing by visually processing and outputtingan image signal that has been input.

Thus, the effect of visual processing can be easily set through aneffect adjustment signal. Consequently, it is possible to adjust theeffect of visual processing in real time.

A 37th aspect of the invention is the 36th aspect of the invention, inwhich the program is for causing a computer to further execute a targetlevel setting step of setting a predetermined target level, and aspatial processing step of performing predetermined spatial processingon an image signal and outputting a processed signal. In the effectadjustment step, a synthesized signal obtained by synthesizing thepredetermined target level and the processed signal according to theeffect adjustment signal for setting the effect of visual processing isoutput. In the visual processing step, tone conversion of an imagesignal is performed based on the synthesized signal that has beensynthesized and the image signal.

Thus, the strength of the visual processing is changed by creatingdifferent synthesized signals with the effect adjustment signal, andthus it is not necessary to change a predetermined tone conversionfunction to match the degree of the strength. Also, by setting a targetlevel in the target level setting step, it is possible to set the toneconversion characteristics to which to change the visual processingeffect that is achieved by the visual processing method.

A 38th aspect of the invention is the 36th aspect of the invention, inwhich the program is for causing a computer to further execute asurrounding image information extraction step of extracting surroundingimage information of an image signal that has been input, and an effectadjustment signal generation step of outputting the effect adjustmentsignal for setting an effect of the visual processing. In the visualprocessing step, an image signal is visually processed based on theimage signal and the surrounding image information. In the effectadjustment step, adjustment is performed to vary the effect of thevisual processing in accordance with the effect adjustment signal.

With this program, it is possible to differ the visual processing effectaccording to the effect adjustment signal.

A 39th aspect of the invention is the 38th aspect of the invention,wherein in the effect adjustment signal generation step, a flat regionthat is adjacent to an edge region is detected from the image signal andthe effect adjustment signal is output.

Thus, additionally, even when an image that has sharp edge regions isinput, it is possible to suppress artifacts in flat regions that arenear edge regions.

A 40th aspect of the invention is the 38th aspect of the invention,wherein the program is for causing a computer to further execute asurrounding image information extraction step of extracting surroundingimage information of an image signal that has been input, and a specialimage detection step of obtaining a degree indicating a degree of aspecial images by detecting statistical bias existing in an image andoutputting the degree as the effect adjustment signal. In the visualprocessing step, an image signal is visually processed based on theimage signal and the surrounding image information. In the effectadjustment step, the effect of the visual processing is set according tothe effect adjustment signal.

With this program, the visual processing effect can be maintained evenin a case where a normal image that is not a special image has beeninput, and artifacts can be inhibited in a case where a special imagehas been input.

A 41st aspect of the invention is an integrated circuit that includesthe visual processing device according to any one of the 1st through29th aspects of the invention.

With this configuration, with an integrated circuit as well it ispossible to obtain the same effect as the visual processing devices.

Effects of the Invention

With the invention, it is possible to achieve a visual processingdevice, a display device, a visual processing method, a program, and anintegrated circuit that can inhibit artifacts even in a case where animage that has sharp edge regions or a special image has been input, andwith a simple configuration can change the strength of the visualprocessing of the image in real time.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a visual processing device according to thefirst embodiment of the invention.

FIG. 2 is a diagram of the characteristics of the target level settingportion of the first embodiment.

FIG. 3 is an explanatory diagram of the unsharp signal US and the outputsignal OS in the first embodiment.

FIG. 4 is an explanatory diagram of the local contrast characteristicsin the first embodiment.

FIG. 5 is an explanatory diagram of the DR compression characteristicsin the first embodiment.

FIG. 6 is a block diagram of a first modified example of the visualprocessing device of the first embodiment.

FIG. 7 is an explanatory diagram of the tone conversion characteristicsin the first embodiment.

FIG. 8 is an explanatory diagram of the gain characteristics in thefirst embodiment.

FIG. 9 is a block diagram showing the configuration of the visualprocessing device of the first embodiment of the invention.

FIG. 10 is an explanatory diagram for describing the two-dimensionaltone conversion characteristics of the same.

FIG. 11 is an explanatory diagram for describing the output of theprocessed signal OS of the same.

FIG. 12 is a block diagram that shows the configuration of the controlsignal generation portion of the same.

FIG. 13 is an explanatory diagram for describing the output of theeffect adjustment signal of the same.

FIG. 14 is a flowchart that describes the operation of that visualprocessing device.

FIG. 15 is a block diagram showing the configuration of the controlsignal generation portion of a modified example of the same.

FIG. 16 is an explanatory diagram for describing the effect adjustmentsignal of a modified example of the same.

FIG. 17 is a block diagram that shows the configuration of a visualprocessing device according to a second embodiment of the invention.

FIG. 18 is a block diagram that shows the configuration of a visualprocessing device according to a third embodiment of the invention.

FIG. 19 is a block diagram that shows the configuration of a visualprocessing system according to a fourth embodiment of the invention.

FIG. 20 is an explanatory diagram for describing the two-dimensionalgain characteristics of the same.

FIG. 21 is a block diagram that shows the configuration of a visualprocessing device according to a sixth embodiment of the invention.

FIG. 22 is an explanatory diagram for describing the two-dimensionaltone conversion characteristics of the same.

FIG. 23 is an explanatory diagram for describing the output of theprocessed signal of the same.

FIG. 24 is a block diagram that shows the configuration of the specialimage detection portion of the same.

FIG. 25 is an explanatory diagram for describing the special images ofthe same.

FIG. 26 is an explanatory diagram for describing edge pixels of thesame.

FIG. 27 is an explanatory diagram for describing the output of theeffect adjustment signal of the same.

FIG. 28 is a flowchart that describes the operation of the visualprocessing device and also is a structural diagram of the continuouschanging portion.

FIG. 29 is a block diagram that shows the configuration of a specialimage detection portion of a first modified example of the same.

FIG. 30 is an explanatory diagram for describing the frequencydistribution that is detected by the frequency detection portion(classifire) of the first modified example of the same.

FIG. 31 is an explanatory diagram for describing the effect adjustmentsignal of the first modified example of the same.

FIG. 32 is a block diagram that shows the configuration of the specialimage detection portion of a second modified example of the same.

FIG. 33 is an explanatory diagram for describing the continuous lengthsof the second modified example of the same.

FIG. 34 is an explanatory diagram for describing the effect adjustmentsignal of the second modified example of the same.

FIG. 35 is a block diagram that shows the configuration of the specialimage detection portion of a third modified example of the same.

FIG. 36 is an explanatory diagram for describing the block images of thethird modified example of the same.

FIG. 37 is an explanatory diagram for describing the effect adjustmentsignal of the third modified example of the same.

FIG. 38 is a block diagram that shows the configuration of a visualprocessing device according to a seventh embodiment of the invention.

FIG. 39 is a block diagram that shows the configuration of a visualprocessing device according to an eighth embodiment of the invention.

FIG. 40 is a block diagram that shows the configuration of a visualprocessing system according to a ninth embodiment of the invention.

FIG. 41 is an explanatory diagram for describing the two-dimensionalgain characteristics of the same.

FIG. 42 is a diagram of the overall configuration of the content supplysystem according to the second embodiment of the invention.

FIG. 43 is a front view of a portable telephone that is provided withthe visual processing device according to the second embodiment.

FIG. 44 is a block diagram for describing the overall configuration ofthe portable telephone according to the second embodiment.

FIG. 45 is an explanatory diagram of the overall configuration of thedigital broadcast system according to the second embodiment.

FIG. 46 is a block diagram that describes an example of a computersystem according to the second embodiment.

FIG. 47 is a block diagram of a conventional visual processing device.

FIG. 48 is a diagram of the tone conversion characteristics of thisdevice.

DESCRIPTION OF THE REFERENCE NUMERALS

-   1, 20, 101, 102, 103, 104, 1001, 1002, 1003, 1004 visual processing    device-   2, 10 spatial processing portion-   3, 21, 30, 31, 32 visual processing portion-   4 target level setting portion-   5, 1020, 1021, 1022, 2021, 2022 effect adjustment portion-   22 multiplication portion-   40 control signal generation portion-   41 edge detection portion-   42 edge proximity detection portion-   43 flatness detection portion-   44, 2144, 704, 84, 93 effect adjustment signal generation portion-   2140, 700, 80, 90 special image detection portion-   50 continuous changing portion-   2141 edge detection portion-   2142 edge amount determination portion-   2143 edge density calculation portion-   2144 effect adjustment signal generation portion-   701 frequency detection portion (classifier)-   702 frequency determination portion-   703 tone level number detection portion-   81 analogous luminance detection portion-   82 continuous length detection portion-   83 mean continuous length calculation portion-   91 high-frequency block detection portion-   92 high-frequency block density detection portion-   1905, 4005 gain-type visual processing device

DETAILED DESCRIPTION OF THE INVENTION

Below, visual processing devices according to embodiments of theinvention are described with reference to the drawings.

First Embodiment

First, the visual processing device according to a first embodiment isdescribed. The visual processing performed here is processing for givingcharacteristics that are close to human vision, and is for determiningan output signal based on the contrast between the value of a targetpixel of an image signal that has been input and the values of thepixels around that target pixel. Examples of processing that may beadopted include backlight correction, knee processing, DR compression,and brightness adjustment (including tone processing and contrastadjustment).

FIG. 1 is a block diagram of a visual processing device 1 according tothe first embodiment.

The visual processing device 1 performs visual processing on an imagesignal IS and outputs an output signal OS that has been visuallyprocessed.

A spatial processing portion 2 obtains the values of pixels targeted forspatial processing, and the values of pixels in the region around thetarget pixels (hereinafter, referred to as “surrounding pixels”), fromthe input signal IS. The spatial processing portion 2 performs spatialprocessing on the input value for each pixel of the original image thathas been obtained and outputs an unsharp signal US. The unsharp signalUS is a blur signal that is obtained by processing the input signal ISwith a low-pass filter.

From the unsharp signal US it is possible to detect the brightness,including the region surrounding the target pixel.

A target level setting portion 4 is for setting a target level for thedesired effect of visual processing. For example, it sets a target forthe tone conversion characteristics up to which to change the effect ofthe local contrast and the desired visual processing effect of DRcompression. The target level setting portion 4 sets a function fordetermining the target level value that has been set according to thetone conversion characteristics to be adjusted, and converts the inputsignal IS based on the predetermined function.

Specifically, for local contrast processing, it outputs a target levelvalue L in accordance with the conversion characteristics of line 1shown in FIG. 2. Here, line 1 changes the target level in accordancewith the input signal IS. For example, the target level L may be setequal to the input signal IS. It should be noted that in this case, thetarget level setting portion 4 is not necessary.

Similarly, for DR compression processing it outputs a target level valueL in accordance with the conversion characteristics of line 2 shown inFIG. 2. Line 2 does not change the target level in accordance with theinput signal IS. In other words, the target level L=a predeterminedvalue T1 (fixed). It should be noted that in this case, the target levelsetting portion 4 is not necessary.

It should be noted that it is also possible to output a target levelvalue L based on conversion characteristics that are intermediate toline 1 and line 2. For example, it is possible for the target levelL=(input signal IS+a predetermined value T1)÷2. Alternatively, it isalso possible to output a target level value L based on a curve 1 thatis set between line 1 and line 2.

An effect adjustment portion 5 synthesizes the target level L and theunsharp signal US by an interpolation computation (“interpolationcomputation” means the calculation of a single physical value from twophysical values through interpolation) in accordance with an outsidesignal (effect adjustment signal) MOD, and outputs a synthesized signal(or “a modified unsharp signal”) MUS. The effect adjustment portion 5for example executes an interpolation computation in whichMUS=(US−L)×MOD+L. The value of the outside signal (effect adjustmentsignal) MOD is set within a range from 0 to 1, where a MOD of 0 is noeffect and a MOD of 1 is a maximum effect. A modified form of theequation is MUS=US×MOD+L×(1−MOD).

The visual processing portion 3 outputs an output signal OS for theinput signal IS and the synthesized signal MUS based on thetwo-dimensional tone conversion characteristics that have been set.

Various visual effects can be achieved by the tone conversioncharacteristics.

Next, the visual processing device 1 of the first embodiment of theinvention is described in further detail.

First, the control in the case of enhancing or weakening the localcontrast as the effect of visual processing is described. Control isperformed by setting an effect adjustment signal from the outside.

The visual processing device 1 is set so that it has the two-dimensionaltone conversion characteristics shown in FIG. 4. Here, the horizontalaxis is the input signal IS that has been input, and the vertical axisis the converted output signal OS.

The two-dimensional tone conversion characteristics are the input/outputcharacteristics of tone conversion for determining an output withrespect to the synthesized signal MUS and the input signal IS. Forexample, it has predetermined tone conversion characteristics accordingto the signal levels from the synthesized signals MUS0 to MUSn in FIG.4. Thus, when the pixel value of the input signal IS is an 8-bit value,the pixel value of the output signal OS corresponding to the value ofthe input signal IS separated into 256 levels is determined based on thepredetermined two-dimensional tone conversion characteristics. The toneconversion characteristics are tone conversion curves that havepredetermined gamma conversion characteristics, and the relationship issuch that the output monotonically decreases along with the subscript ofthe synthesized signal MUS. It should be noted that even if there areranges where the output partially does not monotonically decrease alongwith the subscript of the synthesized signal MUS, it is sufficient forit to be substantially monotonically decreasing. As shown in FIG. 4, inthe two-dimensional tone conversion characteristics, the relationshipwhere (the output value when MUS=MUS0)≧(the output value when MUS=MUS1)≧. . . ≧(the output value when MUS=MUSn) is satisfied for the brightnessvalue of the pixels of all input signals IS. With these tone conversioncharacteristics, the contrast of the local region is enhanced.

Next, the spatial processing portion 2 obtains an unsharp signal US byperforming a low-pass spatial filter computation, which passes only thelow spatial frequencies, on target pixels in the input signal IS. Inthis filter computation, the pixel values of the target pixels and thesurrounding pixels are calculated based on US=(Σ[Wij]×[Aij])/(Σ[Wij]),for example. Here, [Wij] is the weight coefficient of the pixel of thetarget pixel and surrounding pixels that is located in the i-th row j-thcolumn, and [Aij] is the pixel value of the pixel of the target pixeland surrounding pixels that is located in the i-th row j-th column. Thesymbol E means to take the sum for each of the target pixels and thesurrounding pixels.

More specifically, a case in which the weight coefficient [Wij] is 1 andthe pixel value [Aij] is expressed as A(i,j) is described. As for thepixel values of the target pixels, A(1,1) is 128, A(0,0) is 110, A(0,1)is 115, A(0,2) is 117, A(1,0) is 123, A(1,2) is 120, A(2,0) is 120,A(2,1) is 127, and A(2,2) is 125. At this time, to obtain an unsharpsignal US from a region of 3 pixels×3 pixels, the unsharp signal US isobtained by performing the calculationUS=(128+110+115+117+123+120+120+127+125)/9.

It should be noted that it is possible to assign a weight coefficientwith a smaller value the larger the absolute value of the differencebetween the pixel values, and it is also possible to assign a smallerweight coefficient the larger the distance from the target pixels.

The region of the surrounding pixels is a size that is set in advancedepending on the effect. The surrounding pixel region is preferably setto a relatively large region in order to obtain a visual effect. Forexample, when the size of the target image is XGA (1024×768), thesurrounding pixel region is set to a region of at least 80 pixels×80pixels.

As the low-pass spatial filter it is possible to use a FIR (FiniteImpulse Response)-type low-pass spatial filter or an IIR (InfiniteImpulse Response)-type low-pass spatial filter, which are normally usedto create unsharp signals.

In local contrast processing, the target level setting portion 4 setsthe conversion characteristics of line 1 shown in FIG. 2, setting thetarget level L=input signal IS. Consequently, when the effect adjustmentsignal MOD=0, the visual processing has “no effect” and thus thesynthesized signal MUS=the input signal IS.

(for example, if the synthesized signal MUS is found byMUS=US×MOD+IS×(1.0−MOD), then by substituting MOD=0 into this equation,the synthesized signal MUS=the input signal IS.

It should be noted that if the target level L is set equal to the inputsignal IS, then it is not necessary to have the target level settingportion 4. In this case, the input signal IS can be input directly tothe effect adjustment portion 5.

As for the visual processing portion 3, when the synthesized signalMUS=the input signal IS, the two signals that are input to the visualprocessing portion 3, that is, the input signal IS and the synthesizedsignal MUS, are the same value, so that in the visual processing device1, tone conversion based on the tone conversion characteristics of curve2 shown in FIG. 4 is executed. The tone conversion characteristics ofcurve 2 have the characteristics of brightness adjustment only (gammaconversion), and do not have the effect of increasing the localcontrast.

The effect adjustment portion 5 adjusts the effect of the requiredvisual processing based on the setting of the effect adjustment signalMOD. For example, if MOD=0.5, then the synthesized signal MUS is set toMUS=(US−L)×MOD+L, and when L=IS, the synthesized signalMUS=0.5×US+0.5×IS.

As shown in FIG. 3( a), at this time the synthesized signal MUS is anoutput that is intermediate between the input signal IS and the unsharpsignal US. The output signal OS(MUS) that has been visually processedwith the synthesized signal MUS is, as shown in FIG. 3( b), anintermediate output between the output signal OS(IS) that has beenvisually processed based on only the input signal IS and the outputsignal OS(US) that has been visually processed based on only the unsharpsignal US. Thus, the visual processing device 1 outputs an output signalOS(US) with a MOD of 1 whose visual processing is the “maximum effect”and an output signal OS(IS) with a MOD of 0 whose visual processing is“no effect.”

Thus, the visual processing effect of the local contrast can bestrengthened or weakened in accordance with the value of the effectadjustment signal MOD.

The effect adjustment portion 5 also creates a synthesized signal MUS byinterpolating the input signal IS and the unsharp signal US with theeffect adjustment signal MOD. Thus, it can change the effect of thevisual processing from the characteristics of only gamma conversion forconverting a predetermined brightness, to the characteristics forconverting the local contrast.

DR compression processing is described next using FIG. 5. In animage-capturing device such as a camera, DR compression processing isfor keeping the input range of the CCD to which the image is input towithin a recording range for recording by the image-capturing device.Normally, in order to capture an image while adjusting the brightness inreference to a person's face, brightness setting is performed such thatthe brightness of the face becomes about 80% of the DR of the output.Thus, for example, it is necessary for the bright sky region in thebackground of the face, etc., to fit into the remaining 20% of the DR ofthe output. For this reason, generally “knee processing” for compressingand converting the DR of the output is performed from the point that theinput range is equal to or greater than a fixed level, as illustrated bycurve 4 shown in FIG. 5.

However, when the signal of bright regions such as sky is compressed tofit into the range of 20% of the DR of the output, insufficientgradations may lead to the loss of shaded areas (gradation areas) suchas clouds.

Accordingly, as shown in FIG. 5, the visual processing device 1 has tonemapping curves MUS0 to MUSn for converting bright regions whose inputrange has a greater value than approximately 1.0, and based on thesetone mapping curves, controls the degree of compression according to thesynthesized signal MUS. Thus, even if the signal of a bright region isinput to the visual processing device 1, it is possible to prevent alack of gradations in the output signal from occurring.

The conversion characteristics of line 2 in FIG. 2 also may be set inthe target level setting portion 4 so that the target level L=apredetermined value T1 (fixed).

In this case, when the effect adjustment signal MOD is 0 and thesynthesized signal MUS=US×MOD+L×(1.0−MOD), then the synthesized signalMUS becomes T1, and when the effect adjustment signal MOD is 1, thesynthesized signal MUS is equal to the unsharp signal US. A singlepredetermined curve is selected from among the curves MUS0 to MUSn shownin FIG. 5 based on the setting for the target level L.

When the synthesized signal MUS is equal to T1 (fixed value), the visualprocessing portion 3 performs DR compression fixed at the toneconversion curve of curve 3 shown in FIG. 5. When the synthesized signalMUS is equal to the unsharp signal US, it performs DR compression thatcorresponds to the brightness of the local region, and thus in this DRcompression, the effect of the visual processing corresponding to thebrightness of the local region increases.

Next, the effect of the required visual processing is adjusted based onthe setting of the effect adjustment signal MOD by the effect adjustmentportion 5. For example, if MOD=0.5, then the synthesized signalMUS=0.5×US+0.5×T1.

In this way, by setting the effect adjustment signal MOD to apredetermined value, it is possible to achieve DR compression whoseeffects are different strengths, based on a conversion curve that isdetermined through interpolation of the tone conversion curve of curve 3and the tone conversion curve MUSn.

It should be noted that the visual processing device 1 has theconfiguration of outputting the value that is obtained by converting thetone level of the input signal IS with the visual processing portion 3as the output signal OS, but it may also have a configuration in whichit outputs the value of the gain for the input signal IS, whichcorresponds to the value obtained by tone conversion.

FIG. 6 shows a block diagram of a visual processing device 20 accordingto a first modified example. To avoid repetition, the discussion ofprocessing that is identical to that of the visual processing device 1(FIG. 1) has been omitted.

In FIG. 6, the visual processing device 20 has for example been set tothat it has the two-dimensional tone conversion characteristics shown inFIG. 7. Here, the horizontal axis is the input signal IS that has beeninput, and the vertical axis the converted output signal OS.

For the tone conversion characteristics, different tone conversioncurves from MUS0 to MUSn are selected based on the synthesized signalMUS. Based on these characteristics, conversion is performed such thatthe dark regions in the image have a higher contrast due to MUS0 andbecome brighter. On the other hand, conversion of bright regions in theimage is inhibited by MUSn. Thus, it is possible to achieve effectivedark area correction for backlit images in which people's faces are darkand the background region is bright, for example.

Here, a visual processing portion 21 corrects the input signal IS basedon the gain characteristics shown in FIG. 8, using the slope of the toneconversion curve of FIG. 7 as the gain. The horizontal axis is the inputsignal IS that has been input, and the vertical axis is the output ofthe gain signal GAIN.

An output that is equal to the output signal OS shown in FIG. 7 may beobtained by multiplying the gain signal GAIN by the input signal IS. Iftone conversion processing is to be achieved based on the toneconversion curve of FIG. 7, then the target level setting portion 4 setsthe conversion characteristics of line 1 shown in FIG. 2 so that thetarget level L=the input signal IS. Thus, when the effect adjustmentsignal MOD=0, the synthesized signal MUS=the input signal IS.

The effect adjustment portion 5 synthesizes the target level L and theunsharp signal US through interpolation in accordance with the effectadjustment signal MOD, and outputs a synthesized signal MUS. In otherwords, it is possible to adjust the strength of the effect of visualprocessing with the effect adjustment signal MOD.

As for the visual processing portion 21, when the synthesized signalMUS=the input signal IS, the two signals that are input to the visualprocessing portion 3, that is, the input signal IS and the synthesizedsignal MUS, are the same value, and in the visual processing device 20,tone conversion of only the predetermined gamma conversion of curve 5shown in FIG. 7 is executed, and tone conversion that has the effect ofdark area correction is not executed.

Thus, by setting the value of the effect adjustment signal MOD, it ispossible to adjust the effect of tone conversion from tone conversionwith the characteristics of only a predetermined gamma conversion totone conversion for achieving dark area correction.

The visual processing portion 21 outputs a gain signal GAIN based on theinput signal IS and the synthesized signal MUS in order to achieve thegain properties that are set.

A multiplication portion 22 multiplies the gain signal GAIN and theinput signal IS, and outputs an output signal OS.

One feature of the two-dimensional gain characteristics is that thechange in the curve with respect to the input signal IS is smoother thanthe tone conversion curve. Thus, sufficient processing precision can beensured even if the input signal IS and the synthesized signal MUS arethinned out roughly, and the bit precision of the input signal IS thatis input to the visual processing portion 21 can be dropped. Thus, thescale of the circuit can be reduced in the hardware and logic design.

It should be noted that the visual processing portion 21 is constitutedby a two-dimensional lookup table (hereinafter, referred to as “2D LUT”)that gives the relationship between the input signal IS and thesynthesized signal MUS and the gain signal GAIN, and it is also possiblefor a gain signal GAIN to be output with respect to the input signal ISand the synthesized signal MUS in reference to the 2D LUT. Thus, bystoring gain values rather than tone conversion values in the 2D LUT, itis possible to reduce the number of bits of the two input signals, andthis allows the memory capacity to be significantly reduced.

Having a 2D LUT serve as the visual processing portion 21 allows complexgain characteristics to be created in advance. Having a 2D LUT serve asthe visual processing portion 21 allows the visual processing portion 21to be achieved by a read-only memory (hereinafter, “ROM”). To make itpossible to update the gain characteristics, it is also possible for the2D LUT to be constituted by a rewritable memory such as a random accessmemory (hereinafter, “RAM”). The 2D LUT stores gain data havingtwo-dimensional gain characteristics that have been set in advance. Bychanging the two-dimensional gain characteristics, it is possible toobtain various visual effects such as local contrast processing and DRcompression processing.

Further, the visual processing devices 1 and 20 allow the strength ofthe effect to be flexibly adjusted by setting the value of the effectadjustment signal MOD with respect to the strength of the visualprocessing.

It should be noted that the effect adjustment signal MOD is a signalthat can be changed in advance or in real time, depending on thesetting. For example, it may be possible to use a remote control toselect a screen menu of a display device and then set a correspondingvalue as the effect adjustment signal MOD.

It is also possible to automatically extract a feature of the image,such as a predetermined image pattern, image gradation, or color, andautomatically set the most suitably value as the effect adjustmentsignal MOD.

It is also for this to be semi-automatic, such as using a remote controlto make a selection from among “video quality” or “news quality,” forexample, displayed on a screen selection menu of the display device, andthen automatically set a value within an adjustment range that has beenset in advance for each selection menu as the effect adjustment signalMOD.

It is also possible for the visual processing devices 1 and 20 to have abroadcast content detection portion, which is not shown, and for thebroadcast content detection portion to detect genre information orprogram description information of EPG display data that have beenseparated as program information, or genre information or programdescription information of data that are currently being received, andthen to change the effect adjustment signal MOD based on the content ofthe information that has been detected. It should be noted that thegenre information of the data or the information of the image also maybe detected by the MPEG stream information.

As described above, the visual processing device of the embodiment canboth increase and decrease the strength of the visual processing usingan effect adjustment signal MOD, and thus it is not necessary torecreate the tone conversion characteristics data or the gaincharacteristic data in accordance with the strength.

Thus, it is not necessary to provide a dedicated circuit for achievingvisual processing at various strengths or profile data LUTs forachieving visual processing at various strengths. Thus, the visualprocessing device can be achieved with less hardware circuitry and tablememory capacity. Further, the visual processing device uses a LUT and itis not necessary to change the content of the LUT, and thus the timerequired for this change can be obviated and the visual effect to beadjusted in real time.

The visual processing device of this embodiment allows the effectadjustment signal MOD to be changed in real time, and thus the strengthof the visual processing effect can be changed in real time.Specifically, it is possible to change the strength of the visualprocessing in frame or in pixel units, and this allows the strength ofthe visual processing to be changed for local regions in the image.

The effect adjustment portion 5 creates the synthesized signal MUS byinterpolating the unsharp signal US and the input signal IS, the levelsignal, or a preset value for these signals, using the effect adjustmentsignal MOD. Thus, the visual processing device can change the effect ofthe visual processing from characteristics of only gamma conversion fortransforming a predetermined brightness, to characteristics fortransforming the local contrast.

Second Embodiment

Next, a second embodiment of the invention will be described in detailusing the drawings.

In general, natural images have many tone levels, and by performingvisual processing on natural images, it is possible to obtain sharpimages with a high local contrast, for example. On the other hand, whenvisual processing is performed on an image with steep edges, theartifacts are prone to stand out. When the strength of the visualprocessing is reduced in order to suppress the artifacts, the processingbecomes weak even for natural images and the resulting image is notsharp.

Thus, by weakening the visual processing only near edges, it is possibleto maintain the processing effect for the natural image overall whilesuppressing artifacts near the edges.

The visual processing device according to the second embodiment of theinvention performs adjustment (of the strength or correction amount) byoutputting an effect adjustment signal for differing the visualprocessing effect so as to vary the effect of visual processing incorrespondence with the effect adjustment signal.

It also performs adjustment by detecting regions adjacent to an edge orflat regions adjacent to an edge in the image to be subjected to visualprocessing, and creates an effect adjustment signal from the edge amountand the degree of flatness and differs the effect of visual processingin accordance with the effect adjustment signal.

Thus, even if an image with a steep edge region is input to the visualprocessing device, it is possible to obtain the visual processing effectwhile suppressing artifacts near the edge.

Here, visual processing is processing for giving characteristics thatare close to human vision, and is for determining the value of an outputsignal based on the contrast between the value of a target pixel of animage signal that has been input and the values (brightness) of pixelsaround that target pixel. The visual processing may be adopted inbacklight correction, knee processing, DR compression, color processing,and brightness adjustment (including tone processing and contrastadjustment), for example.

In this embodiment of the invention, the luminance component Y or thebrightness component L of a YCbCr color space, a YUV color space, a Labcolor space, a LUV color space, a YIQ color space, or a YPbPr colorspace is defined as the luminance signal. Hereinafter, the luminancesignal is described as the image signal.

The visual processing device of the second embodiment of the inventionis described using FIGS. 9 through 16. FIG. 9 is a block diagram showingthe configuration of a visual processing device 101 according to thesecond embodiment of the invention.

In FIG. 9, the visual processing device 101 according to the secondembodiment of the invention is provided with a spatial processingportion 10 for outputting surrounding image information (unsharp signal)US from an image signal that has been input, a control signal generationportion 40 for outputting an effect adjustment signal MOD in accordancewith the degree of flatness of the edge vicinity region, an effectadjustment portion 1020 for outputting a synthesized signal MUS that issynthesized changing the proportion of the image signal IS and thesurrounding image information US according to the effect adjustmentsignal MOD, and a visual processing portion 30 for visually processingthe image signal IS based on the synthesized signal MUS and the imagesignal IS.

The various functional sections of the visual processing device 101 aredescribed below.

The spatial processing portion 10 extracts the value of a target pixeland the values of pixels in the region around the target pixel(hereinafter, called “surrounding pixels”) from the image signal IS, anduses the values of the pixels that it has extracted to filter the imagesignal IS.

For example, the spatial processing portion 10 performs a low-passfilter on the image signal IS to create an unsharp signal US. Theunsharp signal US is created by a computation such as the following.US=(Σ[Wij]×[Aij])÷(Σ[Wij])

Here, [Wij] is the weight coefficient of the pixel located at the i-throw j-th column among the target pixel and the surrounding pixels, and[Aij] is the pixel value of the pixel located at the i-th row j-thcolumn among the target pixel and the surrounding pixels. The symbol Emeans to calculate the sum for each of the target pixel and thesurrounding pixels.

It should be noted that it is possible to assign a weight coefficientwith a smaller value the larger the absolute value of the differencebetween pixel values, and it is also possible to assign a smaller weightcoefficient the larger the distance from the target pixel. The region ofthe surrounding pixels is a size that is set in advance incorrespondence with the effect, and the visual effect can be increasedby setting this region to size that is larger than a predetermined size.For example, if the size of the target image is 1024 pixels verticallyby 768 pixels horizontally, then by creating an unsharp signal US from aregion that is at least 80 pixels vertically by 80 pixels horizontally,the visual effect can be increased compared to local regions of about 3pixels vertically by 3 pixels horizontally.

A FIR (Finite Impulse Response)-type low-pass spatial filter or an IIR(Infinite Impulse Response)-type low-pass spatial filter, which arecommonly used to create unsharp signals, can be used as the low-passspatial filter.

Next, the effect adjustment portion 1020 synthesizes the image signal ISand the unsharp signal US by interpolation in accordance with the effectadjustment signal MOD that has been output from the control signalgeneration portion 40, and outputs a synthesized signal MUS. Thesynthesized signal MUS is obtained through an interpolation computationsuch as that of Eq. 1 according to the effect adjustment signal MOD. Thecontrol signal generation section 40 is discussed later.MUS=US×MOD+IS×(1.0−MOD)  Eq. 1

Here, the value of the effect adjustment signal MOD is changed withinthe range of 0.0 to 1.0, and when the value of the effect adjustmentsignal MOD is 0.0 there is no effect, whereas when the value of theeffect adjustment signal MOD is 1.0, the strength of the processing is amaximum. It should be noted that Eq. 1 can be modified to Eq. 2, andsimilarly the synthesized signal MUS can be created.MUS=(US−IS)×MOD+IS  Eq. 2

Next, the visual processing portion 30 converts the tone level of theimage signal IS according to the synthesized signal MUS from the effectadjustment portion 1020.

The visual processing portion 30 performs tone conversion based on thetwo-dimensional tone conversion characteristics shown in FIG. 10, forexample. Here, two-dimensional tone conversion refers to tone conversionin which the value of an output is determined with respect to the twoinputs, that is, the synthesized signal MUS and the image signal IS. Thevisual processing portion 30 outputs a processed signal OS with respectto the image signal IS and the synthesized signal MUS based on thetwo-dimensional tone conversion characteristics. It is possible toproduce various visual effects based on the tone conversioncharacteristics.

The two-dimensional tone conversion characteristics are described usingFIG. 10. FIG. 10 is an explanatory diagram for describing thetwo-dimensional tone conversion characteristics. Here, the horizontalaxis is the image signal IS that has been input, and the vertical axisis the processed signal OS that has been transformed.

As shown in FIG. 10, the two-dimensional tone conversion haspredetermined tone conversion characteristics that are in accord withthe signal level of the synthesized signal MUS0 to MUSn. Thus, when thepixel value of the image signal IS is an 8-bit value, the pixel value ofthe output signal OS that corresponds to the value of the image signalIS separated into 256 levels is determined based on the predeterminedtwo-dimensional tone conversion characteristics. The tone conversioncharacteristics are tone conversion curved that have predetermined gammaconversion characteristics, and the relationship is such that the outputmonotonically decreases along with the subscript of the synthesizedsignal MUS. It should be noted that even if there are ranges where theoutput partially does not monotonically decrease along with thesubscript of the synthesized signal MUS, it is sufficient for it to besubstantially monotonically decreasing. As shown in FIG. 10, in thetwo-dimensional tone conversion characteristics, the relationship of(the output value when MUS=MUS0)≧(the output value when MUS=MUS1) ≧ . .. ≧(the output value when MUS=MUSn) is satisfied for the brightnessvalues of all input signal IS pixels.

With the two-dimensional tone conversion characteristics shown in FIG.10, for an input image signal IS with a value “a,” the visual processingportion 30 selects MUS0 when the brightness value of the surroundingregion is small to obtain a value of “P” for the processed signal OS,whereas it selects MUSn when the brightness value of the surroundingregion is large to obtain a value of “Q” for the processed signal OS. Inthis way, even for an input image signal IS with the value “a,” theprocessed signal OS can be significantly varied from the value “P” tothe value “Q” depending the change in the brightness value of thesurrounding region. By doing this, the contrast of dark areas can beenhanced according to the synthesized signal MUS.

On the other hand, if the synthesized signal MUS is made equal to theimage signal IS in order to eliminate the effect of visual processing,then it is possible to adopt the tone conversion characteristics ofcurve 2 shown in FIG. 10. With the tone conversion characteristics ofcurve 2, brightness adjustment (gamma conversion) of the entire image ispossible, but there is no visual effect such as an increase in thecontrast of only local dark area regions.

It should be noted that it is possible to produce various visualprocessing effects by changing the two-dimensional tone conversioncharacteristics, and thus the processing can be adopted for kneeprocessing, DR compression, color processing, or brightness adjustment(including tone processing and contrast adjustment), for example.

Next, in the visual processing portion 30, the processed signal OS whenthe effect of visual processing has been differed based on thesynthesized signal MUS is described using FIG. 11. FIG. 11 is anexplanatory diagram for describing the output of the processed signalOS.

In FIG. 11( a), the horizontal axis is the pixel position to beprocessed, and the vertical axis is the output of the synthesized signalMUS.

For example, when the value of the effect adjustment signal MOD has beenset to 0.5, the synthesized signal MUS becomes an output that isintermediate between the image signal IS and the unsharp signal US.

At this time, as shown in FIG. 11( b), if OS(IS,IS) is a processedsignal OS processed based on only the image signal IS and OS(IS,US) is aprocessed signal OS processed based on the image signal IS and theunsharp signal US, then the processed signal OS(IS,MUS) that is obtainedthrough visual processing according to the image signal IS and thesynthesized signal MUS is an output that is intermediate betweenOS(IS,IS) and OS(IS,US).

Thus, the synthesized signal MUS is equal to US when the value of theeffect adjustment signal MOD is 1.0, and a processed signal OS(IS,US) inwhich the visual processing effect is a maximum is output. On the otherhand, the synthesized signal MUS is equal to IS when the value of theeffect adjustment signal MOD is 0.0, and a processed signal OS(IS,IS) inwhich there is no visual processing effect is output.

In this way, the visual processing portion 30 can strengthen or weakenthe effect of visual processing of the dark area contrast in accordancewith the synthesized signal MUS. Thus, the visual processing device 101can achieve various visual effects of varying effects, from the effectof processing in which only the brightness of the overall image istransformed, to the effect of processing for varying (changing) thecontrast in a local region with the surrounding brightness.

It should be noted that the visual processing device 101 can achieveknee processing, DR compression processing, and color processing, forexample, by changing the two-dimensional tone conversioncharacteristics.

It is also possible for the visual processing portion 30 to have a 2DLUT. In this case, the visual processing portion 30 performs toneconversion by setting the characteristic data (hereinafter, referred toas the “profile”) shown in FIG. 10 in the 2D LUT of the visualprocessing portion 30.

The visual processing portion 30 can also perform visual processingthrough an arithmetic circuit. In particular, if profiles that arecharacteristics that can be approximated by a simple line are set in the2D LUT of the visual processing portion 30, then it is possible toeliminate the table of the 2D LUT and reduce the circuit scale of thevisual processing device 101.

Next, the control signal generation portion 40 is described using FIGS.12 and 13. FIG. 12 is a block diagram showing the configuration of thecontrol signal generation portion 40, and FIG. 13 is an explanatorydiagram for describing the output of the effect adjustment signal MOD.

As shown in FIG. 12, the control signal generation portion 40 isprovided with an edge detection portion 41 for detecting the edgeamount, that is, the luminance difference of each adjacent region, fromthe input signal IS, an edge proximity detection portion 42 fordetecting the degree of proximity of the edge region from the edgeamount, a flatness detection portion 43 for detecting the degree offlatness of flat regions whose luminance difference with an adjacentregion is at or below a predetermined value, and an effect adjustmentsignal generation portion 44 for outputting an effect adjustment signalMOD in accordance with the degree of edge proximity degree and thedegree of flatness.

The edge detection portion 41 detects the edge amount from the imagesignal IS for each region of a predetermined range. The edge detectionportion 41 detects the edge amount EG using an edge detection filter(not shown) such as a first-order derivative filter like a Sobel filteror a Prewitt filter or a second-order derivative filter like a Laplacianfilter. The edge detection portion 41 for example outputs an edge amountsuch as that shown in FIG. 13( b) when the image signal IS shown in FIG.13( a) has been input. Here, in FIG. 13( a) the vertical axis is thevalue of the image signal IS, and the horizontal axis is the pixelposition of the pixel being processed. The vertical axis in FIG. 13( b)is the edge amount EG, and the horizontal axis is the pixel position ofthe pixel being processed.

The edge proximity detection portion 42 detects region near an edge. Forexample, the edge proximity detection portion 42 applies a low-passfilter on the edge amount detected for each predetermined region, andoutputs a degree of proximity that becomes a larger output withincreased edge proximity. For example, as shown in FIG. 13( c), the edgeproximity detection portion 42 outputs a degree of edge proximity thatbecomes a larger output as the edge proximity increases. Here, thevertical axis in FIG. 13( c) is the degree of edge proximity, and thehorizontal axis is the pixel position of the pixel being processed.

The flatness detection portion 43 detects the degree of flatness of flatregions in which the difference in luminance with an adjacent region isat or below a threshold value. For example, as shown in FIG. 13( d), theflatness detection portion 43 detects the difference in luminance withrespect to the adjacent region from the output of the edge amount ofFIG. 13( b), and outputs a larger degree of flatness the smaller thedifference in luminance. Here, the vertical axis in FIG. 13( d) is theflatness FT, which indicates the degree of flatness, and the horizontalaxis is the pixel position of the pixel being processed.

As shown in FIG. 13( e), the effect adjustment signal generation portion44 multiplies the degree of proximity of FIG. 13( c) by the degree offlatness of FIG. 13( d), and outputs an effect adjustment signal MODthat weakens the visual effect the larger the edge proximity degree andthe higher the degree of flatness. Here, the vertical axis in FIG. 13(d) is the output of the effect adjustment signal MOD, and the horizontalaxis is the pixel position of the pixel being processed. The visualeffect by the visual processing device 101 becomes stronger the largerthe value of the effect adjustment signal MOD.

By doing this, the effect adjustment signal generation portion 44, asshown in FIG. 13( e), creates an output that weakens the visual effectin regions that are proximate to an edge, and creates an output thatstrengthens the visual effect in regions that are away from regions thatare near to an edge. Further, in regions that are close to an edge, theeffect adjustment signal generation portion 44, based on the degree offlatness, creates an output that weakens the visual effect the largerthe degree of flatness, and creates an output that strengthens thevisual effect the smaller the degree of flatness.

Thus, with the visual processing device 101, it is possible t ispossible to achieve visual processing in which it is possible to reduceartifacts only near edges, and that has an excellent visual processingeffect for natural images.

Next, the operation of the visual processing device 101 is describedusing FIG. 14. FIG. 14 is a flowchart for describing the operation ofthe visual processing device 101.

As shown in FIG. 16, an image is input to the visual processing device101 (S101), and the edge detection portion 41 detects an edge amount,which is the difference in luminance, for each adjacent region from theimage signal IS (S102).

Next, the edge proximity detection portion 42 of the visual processingdevice 101 processes the edge amount with a low-pass filter to detectthe degree of proximity from the edge amount (S103). The flatnessdetection portion 43 of the visual processing device 101 then detectsthe luminance difference from the edge amount to detect the degree offlatness near the edge (S104).

Next, the effect adjustment signal generation portion 44 of the visualprocessing device 101 multiplies the proximity degree that has beenoutput from the edge proximity detection portion 42 by the flatnessdegree that has been output from the flatness detection portion 43 tocreate an effect adjustment signal MOD (S105).

The visual processing device 101 next, through the effect adjustmentportion 1020, creates a synthesized signal MUS that is synthesizedchanging the ratio of the image signal IS and the unsharp signal US inaccordance with the effect adjustment signal MOD (S106).

Next, the visual processing portion 30 of the visual processing device101 selects a single curve of the two-dimensional tone conversioncharacteristics shown in FIG. 10 based on the synthesized signal MUS,and converts the image signal IS (S107). By doing this, the visualprocessing device 101 executes visual processing that has been adjustedso as to alter the effect of visual processing in accordance with thesynthesized signal MUS.

Next, the visual processing device 101 determines whether or not thereis a pixel to process next (S108). If there are no more pixels for whichprocessing is necessary, then visual processing is completed. On theother hand, if there are further pixels that require processing, thenthe procedure is returned to step 5101 and the next image (pixel) isinput. Thereafter, the steps from S101 to S108 are repeated until thereare no longer any pixels that require processing.

As mentioned above, with the visual processing device 101 of the secondembodiment of the invention, it is possible to achieve visual processingin which it is possible to reduce artifacts only near edges, and thathas an excellent visual processing effect for natural images.

It should be noted that although the visual processing device 101 findsthe degree of edge proximity from the edge amount and finds the degreeof flatness from the input image signal IS, and creates an effectadjustment signal MOD based on the degree of edge proximity and thedegree of flatness, it is also possible to create the effect adjustmentsignal MOD from the amount of change of the unsharp signal US of thespatial processing portion 10.

Below is described a method for detecting a flat region near an edgeaccording to a modified example of the control signal generation portion40.

An implementation in which the effect adjustment signal MOD is createdfrom the amount of change in the unsharp signal US is described usingFIGS. 15 and 16. FIG. 15 is a block diagram showing the configuration ofa control signal generation portion 70.

As shown in FIG. 15, the control signal generation portion 70 isprovided with a change amount detection portion 71 for detecting theamount of change in the unsharp signal US, and an effect adjustmentsignal generation portion 72 for outputting an effect adjustment signalMOD in accordance with the amount of change that has been detected.

The change amount detection portion 71 detects the amount of change inthe unsharp signal US. This detection is performed using an edgedetection filter (not shown) such as a first-order derivative filterlike a Sobel filter or a Prewitt filter or a second-order derivativefilter like a Laplacian filter.

The effect adjustment signal generation portion 72 adjusts the outputaccording to the amount of change that has been detected by the changeamount detection portion 71. That is, the effect adjustment signalgeneration portion 72 outputs an effect adjustment signal MOD with asmaller signal level (value) the higher the amount of change. Forexample, as shown in FIG. 16, the signal level of the effect adjustmentsignal MOD is changed when the amount of change that has been detectedis equal to or greater than a predetermined value Tha, and reduces thesignal level of the effect adjustment signal MOD in the range up to apredetermined value Thb. The signal level of the effect adjustmentsignal MOD is not altered above the predetermined threshold Thb. Bydoing this, it is possible to change the signal level of the effectadjustment signal MOD when a steep edge region has been input, withoutaffecting edge components that are normally present in natural images.Here, the horizontal axis is the amount of change, and the vertical axisis the output (signal level) of the effect adjustment signal MOD. Itshould be noted that a range of 0.0 to 1.0 has been adopted for thesignal level of the effect adjustment signal MOD that is output, but itis also possible to adjust this to from 0.2 to 1.0, for instance,depending on the strength of the visual processing. Further, the visualprocessing device 101 is designed such that the effect of the visualprocessing by the visual processing device 101 becomes stronger thelarger the signal level of the effect adjustment signal MOD.

As illustrated above, with the control signal generation portion 70 itis possible to detect a flat region near an edge and subsequently createthe effect adjustment signal MOD, from the amount of change of theunsharp signal US.

It should be noted that it is also possible for flat regions near edgesto be detected from a reduced image such as a thumbnail image in whichthe image signal has been reduced, and then output an effect adjustmentsignal MOD based on the degree of flatness near the edge or the amountof change in the unsharp signal US.

It is also possible to provide a reducing portion (not shown), forreducing the image signal, at a stage between the input signal and thecontrol signal generation portion 40, and then from the reduced imagethat is produced by the reducing portion, output an effect adjustmentsignal MOD based on the degree of flatness near the edge or the amountof change in the unsharp signal US.

By using a reduced image, it is possible to detect flat regions that arenear edges while suppressing the effects of noise. In other words, thereis less of a noise component in a reduced image that has been created bya reduction method in which an image signal is first averaged and thenthinned out, and thus by using a reduced image, it is possible to detecta flat region near an edge while suppressing the effects of noise.Further, if a reduced signal is used, it is possible to reduce thenumber of pixels that are to be detected, and this allows the number ofcalculations to be reduced.

It is also possible to set a low-pass filter, for instance, before thecontrol signal generation portion 40 or the control signal generationportion 70, to limit the band of the image signal and then detect flatregions that are near edges. By doing this, it is possible to reduce thenoise component, and detect flat regions near edges while suppressingthe effects of noise.

Third Embodiment

In the second embodiment of the invention, a synthesized signal MUS thatis synthesized with different ratios of the image signal IS and thesurrounding image information (unsharp signal) US according to an effectadjustment signal MOD is output, and the visual processing portion 30outputs a processed signal OS that is obtained by visually processingthe image signal IS according to the synthesized signal MUS from theeffect adjustment portion 1020, but in the third embodiment of theinvention, an effect adjustment portion 1021 outputs a synthesizedoutput OUT that is obtained by synthesizing a processed signal OS thathas been visually processed and the image signal IS according to aneffect adjustment signal, and this embodiment is described using FIG.17.

FIG. 17 is a block diagram that shows the configuration of a visualprocessing device 102 according to a third embodiment of the invention.Hereinafter, sections that are identical to those of the secondembodiment are assigned the same reference numerals as before and willnot be described in detail.

In FIG. 17, the visual processing portion 30 outputs a processed signalOS based on the image signal IS and the output US of the spatialprocessing portion 10.

The effect adjustment portion 1021 interpolates the image signal IS andthe processed signal OS in accordance with the effect adjustment signalMOD in order to differ (change) the effect of the visual processing. Forexample, the output OUT from the effect adjustment portion 1021 iscalculated through an interpolation calculation like in Eq. 3 below.OUT=OS×MOD+IS×(1.0−MOD)  Eq. 3

It should be noted that Eq. 3 can be modified as shown in Eq. 4.OUT=(OS−IS)×MOD+IS  Eq. 4

Thus, according to the third embodiment of the invention, it is possibleto output a synthesized signal OUT that is synthesized varying the ratioof the processed signal OS and the image signal IS according to theeffect adjustment signal MOD, and it is possible to differ (change) theeffect of visual processing.

It should be noted that it is also posible to substitute the controlsignal generation portion 70 of the second embodiment of the inventionfor the control signal generation portion 40. In this case as well, itis possible to similarly detect regions that are near an edge and thencreate an effect adjustment signal MOD that corresponds to the amount ofchange in the surrounding information near the edge.

Fourth Embodiment

In the second embodiment of the invention, a synthesized signal MUS thatis synthesized altering the ratio of the image signal IS and thesurrounding image information US according to an effect adjustmentsignal MOD is output, and the visual processing portion 30 outputs aprocessed signal OS that is obtained by visually processing the imagesignal IS according to the synthesized signal MUS from the effectadjustment portion 1020, but in the fourth embodiment of the invention,an effect adjustment portion 1022 outputs a synthesized output OUT thatis synthesized changing the proportion of the outputs of a visualprocessing portion 31 and a visual processing portion 32, which havedifferent visual processing effects, in accordance with the effectadjustment signal MOD, and this embodiment is described using FIG. 18.

FIG. 18 is a block diagram that shows the configuration of a visualprocessing device 103 according to a fourth embodiment of the invention.Hereinafter, sections that are identical to those of the secondembodiment are assigned the same reference numerals as before and willnot be described in detail.

The effect adjustment portion 1022 synthesizes an output OSA of thevisual processing portion 31 in which a first profile 60 has been set inthe LUT and an output OSB of the visual processing portion 32 in which asecond profile 61 has been set in the LUT, which have differentstrengths of visual processing, through an interpolation computation inaccordance with the effect adjustment signal MOD that is output from thecontrol signal generation portion 40, and outputs a synthesized outputOUT. It should be noted that it is also possible to create a synthesizedoutput through an extrapolation computation. At this time, thesynthesized output OUT is as shown in Eq. 5.OUT=OSA×MOD+OSB×(1.0−MOD)  Eq. 5

It should be noted that Eq. 5 can be modified as shown in Eq. 6.OUT=(OSA−OSB)×MOD+OSB  Eq. 6

Thus, according to the visual processing device 103 of the fourthembodiment of the invention, by obtaining a synthesized output that issynthesized varying the ratio of the outputs of the visual processingportion 31 and the visual processing portion 32, which have differentvisual processing effects, in accordance with the effect adjustmentsignal MOD, it is possible to perform visual processing in which thedegree of the visual effect is differed.

It should be noted that it is also possible to substitute the controlsignal generation portion 70 of the second embodiment of the inventionfor the control signal generation portion 40. In this case as well, itis possible to similarly detect regions that are near an edge and thencreate an effect adjustment signal MOD that corresponds to the amount ofchange in the surrounding information near the edge.

Fifth Embodiment

In the visual processing devices of the second embodiment of theinvention through the fourth embodiment of the invention, a toneconversion value that is based on two-dimensional tone conversioncharacteristics is output, but the fifth embodiment of the inventiondescribes a case in which tone conversion is performed using a gainsignal using FIGS. 19 and 20.

FIG. 19 is a block diagram that shows the configuration of a gain-typevisual processing system 104 according to the fifth embodiment of theinvention, and FIG. 20 is an explanatory diagram for describing thetwo-dimensional gain characteristics. Hereinafter, sections that areidentical to those of the second embodiment are assigned the samereference numerals as before and will not be described in detail.

In FIG. 19, the gain-type visual processing system 104 is provided witha gain-type visual processing device 1905 for outputting a gain signalGAIN that is obtained by visually processing the image signal IS, and amultiplier 1911 for multiplying the gain signal GAIN and the imagesignal IS.

The gain-type visual processing device 1905 is provided with the visualprocessing device 101 for outputting a processing signal OS that isobtained by visually processing the image signal IS, and a divider 1912for dividing the processed signal OS by the image signal IS. Here, thevisual processing device 101 outputs a tone conversion value that isobtained by visually processing the output of the image signal IS, andby dividing this tone conversion value by the image signal IS, it ispossible to achieve the gain-type visual processing device 5.

The multiplier 1911 multiplies the image signal IS and the gain signalGAIN that is output by the gain-type visual processing device 1905, andoutput a tone conversion value produced by visually processing theoutput of the image signal IS.

It should be noted that it is also possible for the visual processingportion 30 to carry out processing by directly using a profile that hasthe two-dimensional gain characteristics shown in FIG. 20. Here, thevertical axis of FIG. 20 is the gain output GN, and the horizontal axisis the image signal IS. The two-dimensional gain characteristics shownin FIG. 20 are equivalent to those that are obtained by dividing theoutput of the profile of the two-dimensional tone conversioncharacteristics shown in FIG. 10 by the image signal IS. It is alsopossible to set a profile that has these two-dimensional gaincharacteristics in the LUT of the visual processing portion 30 of thevisual processing device 101. If a profile of the two-dimensional gaincharacteristics is set in the LUT of the visual processing portion 30 inadvance, then the gain signal GN and the gain signal GAIN become equal,and thus it is possible to achieve the gain-type visual processingdevice 1905 without the divider 12.

With the gain-type visual processing device 1905, there is little changein the processed signal with respect to the change in the image signalIS that has been input, and thus it is possible to reduce the bit numberof the input signal and also to reduce the circuit scale. Additionally,if the visual processing portion 30 is provided with a 2D LUT, then itis possible to reduce the memory capacity.

Thus, with the gain-type visual processing system 104 of the fifthembodiment of the invention, by controlling the gain signal GAIN it ispossible to easily suppress saturation of the tone and achieve excellentvisual processing.

It should be noted that the visual processing device 101 of the secondembodiment of the invention can be replaced with the visual processingdevice 102 of the third embodiment of the invention. The gain-typevisual processing device 1905 can be similarly achieved in this case aswell.

Likewise, the visual processing device 101 of the second embodiment ofthe invention can be replaced with the visual processing device 103 ofthe fourth embodiment of the invention. The gain-type visual processingdevice 1905 can be similarly achieved in this case as well.

Thus, according to the second embodiment of the invention through thefifth embodiment of the invention, it is possible to achieve visualprocessing in which artifacts are suppressed, even if an image that hassteep edge regions has been input.

It should be noted that the visual processing device described in thisembodiment can be provided in or connected to a device for handlingmoving images, and may create an effect adjustment signal MOD from theimage of each frame or each field. The control signal generation portion40 can extract edge information or flatness information from a frameimage at least one (frame) prior when the image signal is a frame image,or from a field image at least one (field) prior when the image signalis a field image. By doing this, the visual processing device can use aneffect adjustment signal MOD that corresponds to the edge information orthe flatness information from the top of the frame. It is also possiblefor the visual processing device to extract edge information or flatnessinformation from a field image at least one (field) prior, and use aneffect adjustment signal MOD that corresponds to the edge information orthe flatness information from the top of the field image. It is alsopossible for the control signal generation portion 40 to extract edgeinformation or flatness information from a frame image at least one(frame) prior or from a field image at least one (field) prior, and bydoing this it becomes easy to coordinate the delay of the circuit andthe circuit scale can be reduced.

Sixth Embodiment

In general, natural images have a large number of tone levels, and byperforming visual processing on a natural image it is possible to obtaina sharp image in which the local contrast, for instance, has beenenhanced. On the other hand, special images have a statistical bias,such as either an extremely low proportion of regions in which thegradation changes in the image of the image signal, or an extremelylarge proportion of regions in which the gradation does not change inthe image of the image signal. In such special images, there are manyflat regions in the image. For this reason, artifacts easily stand outwhen visual processing is executed on a special image with steep edges.Weakening the visual processing in order to suppress these artifactsweakens the processing for natural images as well and results in imagesthat are not sharp.

Thus, by suppressing artifacts only for special images, it is possibleto achieve an excellent visual processing effect for natural images.

The visual processing device in the sixth embodiment of the inventiondetects special images that have a statistical bias from the imagesignal, creates an effect adjustment signal based on the degree of thestatistical bias, and then performs adjustment so as to differ (change)the effect of visual processing in accordance with the effect adjustmentsignal that has been created.

Here, the visual processing is processing for giving characteristicsthat are close to human vision, and is processing for determining thevalue of an output signal based on the contrast between the value of atarget pixel of an image signal that has been input and the values ofpixels around that target pixel. The visual processing may be adopted inbacklight correction, knee processing, DR compression, color processing,and brightness adjustment (including tone processing and contrastadjustment).

In this embodiment of the invention, the luminance component Y or thebrightness component L of a YCbCr color space, a YUV color space, a Labcolor space, a Luv color space, a YIQ color space, or a YPbPr colorspace is defined as the luminance signal. Hereinafter, the luminancesignal is described as the image signal.

The visual processing device of the sixth embodiment of the invention isdescribed using FIGS. 21 through 28. FIG. 1 is a block diagram thatshows the configuration of a visual processing device 1001 of the sixthembodiment of the invention.

In FIG. 21, the visual processing device 1001 of the sixth embodiment ofthe invention is provided with a spatial processing portion 10 forextracting surrounding image information (unsharp signal) US from theimage signal IS that has been input, a special image detection portion2140 that detects special images that have a statistical bias from theimage signal IS and output a special image effect adjustment signal DSfor differing the effect of visual processing based on the degree of thestatistical bias, a continuous changing portion 50 for outputting aneffect adjustment signal MOD that is obtained by continuously changingthe special image effect adjustment signal DS between frames, an effectadjustment portion 1020 for outputting a synthesized signal MUS that issynthesized changing the proportion of the image signal IS and thesurrounding image information US according to the effect adjustmentsignal MOD, and a visual processing portion 30 for outputting aprocessed signal OS that is obtained by visually processing the imagesignal IS according to the synthesized signal MUS from the effectadjustment portion 1020.

With this configuration, the special image detection portion 2140 canoutput a special image effect adjustment signal DS that corresponds tothe degree of the information bias held by the special image. The effectadjustment portion 1020 can create a synthesized signal MUS fordiffering the effect of visual processing by the visual processingportion 30, using a effect adjustment signal MOD that is obtained bycontinuously changing the special image effect adjustment signal DS. Thevisual processing portion 30 can convert the tone level of the imagesignal IS according to the synthesized signal MUS that is output fromthe effect adjustment portion 1020.

Thus, even if a special image has been input, the visual processingdevice 1001 can detect the special image and the visual processingportion 30 can differ the effect of visual processing for the specialimage in order to suppress artifacts.

The functional portions of the visual processing device 1001 aredescribed below.

The spatial processing portion 10 extracts the value of a target pixeland the values of pixels in the region around the target pixel(hereinafter, called “surrounding pixels”) from the image signal IS, anduses the values of the pixels that it has extracted to perform filterprocessing on the image signal IS.

For example, the spatial processing portion 10 performs a low-passfilter on the image signal IS to create an unsharp signal US from theimage signal IS. The unsharp signal US is created by a computation suchas the following.US=(Σ[Wij]×[Aij])/(Σ[Wij])

Here, [Wij] is the weight coefficient of the pixel located at the i-throw j-th column among the target pixel and the surrounding pixels, and[Aij] is the pixel value of the pixel located at the i-th row j-thcolumn among the target pixel and the surrounding pixels. The symbol Σmeans to calculate the sum for the target pixel and the surroundingpixels.

It should be noted that it is possible to assign a weight coefficientwith a smaller value the larger the absolute value of the differencebetween pixel values, and it is also possible to assign a smaller weightcoefficient the larger the distance from the target pixel. The region ofthe surrounding pixels is a size that is set in advance according to theeffect, and the visual effect can be increased by setting this region tosize that is larger than a predetermined size. For example, if the sizeof the target image is 1024 pixels vertically by 768 pixelshorizontally, then by creating an unsharp signal US from a region thatis at least 80 pixels vertically by 80 pixels horizontally, the visualeffect can be increased compared to each local region of about 3 pixelsvertically by 3 pixels horizontally.

A spatial filter such as a FIR (Finite Impulse Response)-type low-passspatial filter or an IIR (Infinite Impulse Response)-type low-passspatial filter can be used as the low-pass filter.

Next, the effect adjustment portion 1020 synthesizes the image signal ISand the unsharp signal US by interpolation in accordance with the effectadjustment signal MOD that has been output from the continuous changingportion 50, and outputs a synthesized signal MUS. The synthesized signalMUS is obtained through an interpolation computation such as Eq. 7 inaccordance with the effect adjustment signal MOD. The continuouschanging portion 50 is described later.MUS=US×MOD+IS×(1.0−MOD)  Eq. 7

Here, the value of the effect adjustment signal MOD changes within therange of 0.0 to 1.0, with no visual processing effect when the value ofthe effect adjustment signal MOD is 0.0, and a maximum visual processingeffect when it is 1.0. It should be noted that Eq.7 can be modified toEq. 8, and the synthesized signal MUS can be similarly created.MUS=(US−IS)×MOD+IS  Eq. 8

Next, the visual processing portion 30 converts the tone level of theimage signal IS in accordance with the synthesized signal MUS from theeffect adjustment portion 1020.

The visual processing portion 30 performs tone conversion based on thetwo-dimensional tone conversion characteristics shown in FIG. 22, forexample. Here, two-dimensional tone conversion refers to tone conversionin which the value of the output is determined with respect to the twoinputs of the synthesized signal MUS and the image signal IS. The visualprocessing portion 30 outputs a processed signal OS with respect to theimage signal IS and the synthesized signal MUS based on thetwo-dimensional tone conversion characteristics. Various visual effectscan be produced with the tone conversion characteristics.

The two-dimensional tone conversion characteristics shall be describedusing FIG. 22. FIG. 22 is an explanatory diagram for describing thetwo-dimensional tone conversion characteristics. Here, the horizontalaxis is the image signal IS that has been input, and the vertical axisis the output of the converted processed signal OS.

As shown in FIG. 22, two-dimensional tone conversion has predeterminedtone conversion characteristics according to the signal level of thesynthesized signals MUS0 through MUSn. Thus, when the pixel value of theimage signal IS is an 8-bit value, the pixel value of the output signalOS with respect to the value of the input signal IS separated into 256levels is determined based on the predetermined two-dimensional toneconversion characteristics. The tone conversion characteristics are toneconversion curves that have predetermined gamma conversioncharacteristics, and have the relationship where the outputmonotonically decreases along with the subscript of the synthesizedsignal MUS. It should be noted that even if there are ranges where theoutput partially does not monotonically decrease along with thesubscript of the synthesized signal MUS, it is sufficient for the outputto be substantially monotonically decreasing. As shown in FIG. 22, thetwo-dimensional tone conversion characteristics satisfy the relationshipof (the output value when MUS=MUS0) (the output value when MUS=MUS1)≧ .. . ≧(the output value when MUS=MUSn) with respect to the brightnessvalue of all image signals IS.

According to the two-dimensional tone conversion characteristics shownin FIG. 22, for an image signal IS with a value of “a” that has beeninput, the visual processing portion 30 selects MUS0 when the brightnessvalue of the surrounding region is small to obtain a value “P” for theprocessed signal OS, and selects MUSn when the brightness value of thesurrounding region is large in order to obtain a value “Q” for theprocessed signal OS. In this way, even for an input image signal IS withthe value “a,” the processed signal OS can be significantly changed fromthe value “P” to the value “Q” depending the change in the brightnessvalue of the surrounding region. By doing this, the contrast of darkareas can be enhanced according to the synthesized signal MUS.

On the other hand, if the synthesized signal MUS is made equal to theimage signal IS in order to eliminate the effect of visual processing,then it is possible to have the tone conversion characteristics of curve2 shown in FIG. 22. With the tone conversion characteristics of curve 2,brightness adjustment (gamma conversion) of the entire image ispossible, but there is no visual effect such as an increase in the localcontrast.

It should be noted that it is possible to produce various visualprocessing effects by changing these two-dimensional tone conversioncharacteristics, and thus the visual processing can be adopted for kneeprocessing, DR compression, color processing, or brightness adjustment(including tone processing and contrast adjustment).

Next, he processed signal OS when the effect of visual processing hasbeen differed based on the synthesized signal MUS by the visualprocessing portion 30 is described using FIG. 23. FIG. 23 is anexplanatory diagram for describing the output of the processed signalOS.

In FIG. 23( a), the horizontal axis is the pixel position to beprocessed, and the vertical axis is the output of the synthesized signalMUS.

For example, when the value of the effect adjustment signal MOD has beenset to 0.5, the synthesized signal MUS becomes an output that isintermediate between the image signal IS and the unsharp signal US.

At this time, as shown in FIG. 23( b), if OS(IS,IS) is a processedsignal OS that has been processed based on only the image signal IS andOS(IS,US) is a processed signal OS that has been processed based on theimage signal IS and the unsharp signal US, then a processed signalOS(IS,MUS) that has been obtained by visual processing based on theimage signal IS and the synthesized signal MUS is an output that isintermediate between OS(IS,IS) and OS(IS,US).

Thus, the synthesized signal MUS is equal to US when the value of theeffect adjustment signal MOD is 1.0, and a processed signal OS(IS,US) inwhich the visual processing effect is a maximum is output. On the otherhand, the synthesized signal MUS is equal to IS when the value of theeffect adjustment signal MOD is 0.0, and a processed signal OS(IS,IS) inwhich there is no visual processing effect is output.

In this way, the visual processing portion 30 can strengthen or weakenthe effect of the enhancing the local contrast according to thesynthesized signal MUS. Thus, the visual processing device 1001 canachieve various visual effects that are different effects, from theeffect of processing in which only the brightness of the overall imageis transformed, to the effect of processing in which the contrast inlocal regions is varied (changed) with the surrounding brightness.

It should be noted that the visual processing device 1001 can achieveknee processing, DR compression processing, and color processing, forexample, by changing the two-dimensional tone conversioncharacteristics.

It is also possible for the visual processing portion 30 to have a 2DLUT. In this case, the visual processing portion 30 performs toneconversion by setting the characteristic data (hereinafter, referred toas the “profile”) shown in FIG. 22 in the 2D LUT of the visualprocessing portion 30.

The visual processing portion 30 can also perform visual processingthrough an arithmetic circuit. In particular, if the 2D LUT of thevisual processing portion 30 is provided with profiles, which arecharacteristics that can be approximated by a simple line, then it ispossible to eliminate the table of the 2D LUT and reduce the circuitscale of the visual processing device 1001.

Next, the special image detection portion 2140 is described using FIGS.24 through 27. Here, a case in which the bias of the information of thespecial image is detected from the proportion of regions in which thegradation changes in the image. The change in the gradation is detectedfrom the edge component.

FIG. 24 is a block diagram showing the configuration of the specialimage detection portion 2140, FIG. 25 is an explanatory diagram fordescribing the special image, FIG. 26 is an explanatory diagram fordescribing the edge pixels, and FIG. 27 is an explanatory diagram fordescribing the output of the special image effect adjustment signal DS.

As shown in FIG. 24, the special image detection portion 2140 isprovided with an edge detection portion 2141 for detecting an edgeamount for each pixel from the image signal IS, an edge amountdetermination portion 2142 for determining an edge pixel in which theedge amount is equal to or greater than a predetermined value, an edgedensity calculation portion 2143 for calculating the ratio of the numberof edge pixels to the total number of pixels in the image signal IS, andan effect adjustment signal generation portion 2144 for outputting aspecial image effect adjustment signal DS according to the ratio thathas been calculated by the edge density calculation portion 2143.

Thus, with the visual processing device 1001 it is possible to detectspecial images with an extremely small tone level number in which theedge component is restricted to the border region of a drawing image,and the bias of that information can be detected.

The special image detection portion 2140 detects a statistical bias froma frame image one or more frames prior when the image signal is a frameimage, and detects a statistical bias from a field image one or morefields prior when the image signal is a field image. By doing this, thevisual processing device 1001 can use a special image effect adjustmentsignal DS that corresponds to the bias of the information of the specialimage from the top of the frame or the field.

For example, a case in which the special image detection portion 2140processes the special image 200 shown in FIG. 25 is described. As shownin FIG. 25, the special image 200 has a background region 201, a patterngroup 202, a pattern group 203, and a pattern group 204, and each one ofthese regions has a tone level (gradation value) that is constant orthat fluctuates little. Each group is made of different shapes whosetone level (gradation value) is substantially the same.

The edge detection portion 2141 detects the edge amount for each pixelfrom the image signal IS. The edge detection portion 2141 detects theedge amount using an edge detection filter (not shown) such as afirst-order derivative filter like a Sobel filter or a Prewitt filter ora second-order derivative filter like a Laplacian filter.

The edge amount determination portion 2142 compares the edge amount anda threshold value that has been set in advance for each pixel, anddetermines that a pixel is an edge pixel when the edge amount is equalto or greater than the predetermined threshold value.

For example, due to the processing of the special image 200 by the edgeamount determination portion 2142, an output 300 such as that shown inFIG. 26 is obtained.

In FIG. 26, the edge pixels are the edge pixels 301, the edge pixels302, and the edge pixels 303, and occur in the border region of thegraphic patterns of the special image 200.

Next, returning to FIG. 24, the edge density calculation portion 2143calculates the edge density, which is the ratio of the number of edgepixels to the total number of pixels in the image signal IS, as follows.edge density=edge pixel number÷total pixel number

Here, if the image signal IS is a frame image, then the edge density isthe ratio of the edge pixel number to the total pixel number in theframe. If the image signal IS is a field image, then the edge density isthe ratio of the edge pixel number to the total pixel number in thefield.

The effect adjustment signal generation portion 2144 adjusts the outputaccording to the edge density. In other words, the effect adjustmentsignal generation portion 2144 outputs a special image effect adjustmentsignal DS with a larger signal level (value) the larger the edgedensity. For example, as shown in FIG. 27, it increases the signal levelof the special image effect adjustment signal DS when the edge densityis in the range from a predetermined value Tha to a predetermined valueThb. By setting threshold values in this way, it is possible to create aspecial image effect adjustment signal DS in which the visual effect hasbeen completely eliminated if the edge density is below the thresholdvalue Tha, which is included in special images. On the other hand, if apixel is greater than a threshold value Yhb, which is included in normalimages that are not special images, it is possible to create a specialimage effect adjustment signal DS for processing without weakening thevisual effect. Here, the horizontal axis is the edge density, and thevertical axis is the output of the special image effect adjustmentsignal DS. It should be noted that the range of the signal level of thespecial image effect adjustment signal DS has been set from 0.0 to 1.0,but it is also possible to adjust this to 0.2 to 1.0, for instance, inaccordance with the strength of the visual processing. The visualprocessing device 1001 is configured such that the effect of visualprocessing becomes stronger the larger the signal level of the specialimage effect adjustment signal DS.

The continuous changing portion 50 operates to continuously change theeffect adjustment signal MOD between frames when the special imageeffect adjustment signal DS is output in frame units, or between fieldswhen the special image effect adjustment signal DS is output in fieldunits. For example, the continuous changing portion 50 is provided witha memory portion (not shown) such as a register for temporarily storingthe special image effect adjustment signal DS, and creates the effectadjustment signal MOD by interpolating the special image effectadjustment signal DS that is output from the special image detectionportion 2140 in a new frame and the special image effect adjustmentsignal DS that has been stored temporarily, and this effect adjustmentsignal MOD that is created is stored in the memory portion. The memoryportion stores the first special image effect adjustment signal DS thatis detected as an initial value. The continuous changing portion 50outputs the effect adjustment signal MOD that is created through thisinterpolation computation. By doing this, the effect adjustment signalMOD is kept from changing abruptly between frames. The continuouschanging portion 50 can be achieved by an IIR-type filter, for example.

Next, the operation of the visual processing device 1001 is describedusing FIG. 28. FIG. 28( a) is a flowchart for describing the operationof the visual processing device 1001. FIG. 28( b) is a diagram thatshows an example of the configuration of the continuous changing portion50.

As shown in FIGS. 28( a) and (b), if the image signal IS is a frameimage, then in order to detect a statistical bias from the frame imageone or more frames prior, a plurality of frame images are input to thevisual processing device 1001. Alternatively, if the image signal IS isa field image, then in order to detect a statistical bias from the fieldimage one or more fields prior, a plurality of field images are input tothe visual processing device 1001 (S201). Once a plurality of frameimages or a plurality of field images have been input to the visualprocessing device 1001, the special image detection portion 2140 detectsa special image from the image signal IS, which is a frame image or afield image to be detected, and outputs a special image effectadjustment signal DS that corresponds to the statistical bias of thespecial image that has been detected (S202).

Next, the visual processing device 1001 performs interpolation such thatthe effect adjustment signal MOD is continuously changing betweenframes. The visual processing device 1001 reads the effect adjustmentsignal MOD1 of one frame prior, which has been temporarily stored in amemory portion 5001 such as a register for temporary storage by thecontinuous changing portion 50 (S203), and the special image effectadjustment signal DS that was detected in step 5202 and the effectadjustment signal MOD1 that was read in step 5203 are interpolated by aninterpolation computation, for example, and the effect adjustment signalMOD that is created by this interpolation processing is output from thecontinuous changing portion 50 (S204). Thus, sudden changes that occurbetween processed frame images are suppressed, and it is possible tosuppress flickering of the image, for example, that results fromdifferences in the visual effect.

Next, the visual processing device 1001 temporarily stores the effectadjustment signal MOD that has been created by interpolating the specialimage effect adjustment signal DS and the effect adjustment signal MOD1in the memory portion 5001 (S205). If the interpolation processing isthe result of an internal division computation, then the ratio of thatinterpolation can be given in advance.

Next, the effect adjustment portion 1020 of the visual processing device1001 creates a synthesized signal MUS by synthesizing the image signalIS and the unsharp signal US from the spatial processing portion 10 inaccordance with the effect adjustment signal MOD (S206).

The visual processing portion 30 of the visual processing device 1001then selects one of the curves of the two-dimensional tone conversioncharacteristics shown in FIG. 22 and transforms the image signal ISaccording to the synthesized signal MUS (S207).

Next, the visual processing device 1001 determines whether or not thereis a frame image to process next (S208). If there are no more frameimages that require processing, then the visual processing is completed.On the other hand, if there are frame images that require processing,then the procedure is returned to step 5201 and the next frame image isinput. Thereafter, the steps from S201 to S208 are repeated until thereare no longer any frames that require processing.

It should be noted that above, a case in which interpolation processingis performed so to continuously change the effect adjustment signal MODbetween frames, but the target for interpolation processing is notlimited to between frames, and it may also be between fields as well.

As discussed above, with the visual processing device 1001 of the sixthembodiment of the invention, even if a special image has been input, theedges in the image are detected and the effect of visual processing isadjusted based on the edge amount that has been detected, and thus it ispossible to increase the visual effect in natural images whilesuppressing artifacts in special images.

It should be noted that the method of detecting a statistical bias isnot limited to the method of the special image detection portion 2140discussed above. Special images have a statistical bias, such as eitheran extremely low proportion of regions in which the gradation changes inthe image of the image signal IS, or an extremely large proportion ofregions in which the gradation does not change in the image of the imagesignal IS.

Another modified example of the method of detecting a statistical biasis described below.

First, a special image detection portion 700 according to a firstmodified example is described. With the special image detection portion700 according to the first modified example, a statistical bias isdetected from the proportion of regions in which the gradation does notchange in the image of the image signal IS. Regions in which thegradation does not change can be detected from the degree of flatness ofthe image. A method of detecting a bias in the number of tone levelsfrom the image signal IS is adopted as the method for detecting thedegree of flatness. In images in which the there are very few tonelevels (number of tones) that can be taken for the pixels making up theimage (images in which there is an extremely narrow distribution of thenumber of tone levels taken by the pixels), there is a wide region inwhich the gradation is constant, and thus the degree of flatness in theimage becomes higher. The degree of the special image can be found fromthis bias in information.

Using FIGS. 29, 30, and 31, a first modified example of a case in whicha bias in the tone level number is detected from the image signal IS isdescribed. FIG. 29 is a block diagram showing the configuration of thespecial image detection portion 700 of the first modified example, FIG.30 is an explanatory diagram for describing the frequency distributionthat is detected by a frequency distribution portion 701 of the firstmodified example, and FIG. 31 is an explanatory diagram for describingthe special image effect adjustment signal DS that is output from thespecial image detection portion 700 of the first modified example.

As shown in FIG. 29, the special image detection portion 700 is providedwith a frequency detection portion (classifier) 701 for detecting thefrequency of each tone level from the image signal, a frequencydetermination portion 702 for comparing the frequency of each tone levelwith a predetermining threshold and determining whether the tone levelhas a higher frequency than the predetermined threshold, a tone levelnumber detection portion 703 for detecting the number of tone levelsthat have been determined to be high frequency by the frequencydetermination portion 702, and an effect adjustment signal generationportion 704 for outputting an effect adjustment signal in accordancewith the number of tone levels that has been detected by the tone levelnumber detection portion 703.

The frequency detection portion (classifier) 701 detects the frequencyof each tone level from the image signal using a histogram method. Forexample, if the image signal has 256 tone levels, then it detects thefrequency with which the tone levels appear from 0 to 255.

The frequency determination portion 702 compares the frequency of eachtone level with a predetermining threshold to detect tone levels with ahigher frequency than the predetermined threshold.

As shown in FIG. 30, the frequency determination portion 702 determinesthat a frequency 401 is larger than a predetermined threshold Th at atone level La. Similarly, the frequency determination portion 702determines that a frequency 402, a frequency 403, and a frequency 400are each larger than a predetermined threshold Th at the tone levels Lb,Lc, and Ld. Here, the horizontal axis in FIG. 30 is the tone level, andthe vertical axis is the frequency.

The tone level number detection portion 703 calculates the number oftone levels that have been determined to be high frequency by thefrequency determination portion 702.

Based on the number of tone levels that has been calculated, the effectadjustment signal generation portion 704 increases the signal level(value) of the special image effect adjustment signal DS the larger thetone level number and outputs the special image effect adjustment signalDS. For example, as shown in FIG. 31, the signal level (value) of thespecial image effect adjustment signal DS is increased over the range ofa calculated tone level number of the predetermined value Thc to thepredetermined value Thd.

By providing threshold values in this way, it is possible for the effectadjustment signal generation portion 704 to create a special imageeffect adjustment signal DS for eliminating the visual effect completelyif the tone level number is equal to or below the threshold Thc, whichis included in special images. On the other hand, the effect adjustmentsignal generation portion 704 can create a special image effectadjustment signal DS for processing without weakening the visual effectif the tone level number is equal to or greater than a threshold valueThd, which is included in normal images that are not special images.Here, the horizontal axis is the tone level number, and the verticalaxis is the output of the special image effect adjustment signal DS. Itshould be noted that the range of the value of the special image effectadjustment signal DS that is output has been set from 0.0 to 1.0, but itis also possible to adjust this to from 0.2 to 1.0, for instance, inaccordance with the strength of the visual processing. The visualprocessing device 1001 is configured such that the effect of the visualprocessing becomes stronger the larger the value of the special imageeffect adjustment signal DS.

Thus, with the special image detection portion 700 of the first modifiedexample, it is possible to detect the degree of a special image from theimage signal based on the bias of the image information, and the specialimage detection portion 2140 can be substituted with the special imagedetection portion 700.

Next, a special image detection portion 80 according to a secondmodified example is described. With the special image detection portion80 according to the second modified example, the statistical bias isdetected from the proportion of regions in which the gradation does notchange in the image of the image signal IS. Regions in which thegradation does not change can be detected by the degree of flatness ofthe image. As the method for detecting the degree of flatness, a methodin which a continuous length of analogous pixels whose difference inluminance with adjacent pixels is below a predetermined value isdetected from the image signal IS, and then a mean continuous lengthobtained by taking the mean of a plurality of continuous lengths thathave been detected is adopted. By doing this, it is possible to detectthe degree of the special image. In special images, there are broadregions of constant gradation, and thus the degree of flatness in theimage is high and many pixels with an analogous luminance follow oneother. In other words, it is possible to detect the degree of a specialimage from the statistical bias.

The case of the second modified example, in which continuous lengthswhen analogous luminance signals are continuous are detected from theimage signal, is described using FIGS. 32, 33, and 34.

FIG. 32 is a block diagram showing the configuration of the specialimage detection portion 80 of the second modified example, FIG. 33 is anexplanatory diagram for describing the continuous lengths of the secondmodified example, and FIG. 34 is an explanatory diagram for describingthe special image effect adjustment signal DS of the second modifiedexample.

As shown in FIG. 32, the special image detection portion 80 of thesecond modified embodiment is provided with an analogous luminancedetection portion 81 for detecting analogous pixels whose difference inluminance with adjacent pixels is less than a predetermined value fromthe image signal IS, a continuous length detection portion 82 fordetecting a continuous length of contiguous analogous pixels, a meancontinuous length calculation portion 83 for calculating a meancontinuous length by finding the mean of a plurality of continuouslengths that have been detected by the continuous length detectionportion 82, and an effect adjustment signal generation portion 84 foroutputting a special image effect adjustment signal DS according to themean continuous length.

The analogous luminance detection portion 81 detects analogous pixelswhose difference in luminance with adjacent pixels is equal to or lessthan a predetermined value from the image signal. The predeterminedvalue is a value that is found experimentally in advance, and isdetermined by the picture quality specifications of a device inquestion.

The continuous length detection portion 82 detects continuous lengths ofcontiguous analogous pixels. For example, as shown in FIG. 33, multiplepixels of continuous analogous pixels are detected as continuous lengthsin the vertical direction, such as the vertical direction 503, thevertical direction 504 and the vertical direction 505, and in thehorizontal direction, such as the horizontal direction 500, thehorizontal direction 501 and the horizontal direction 502.

The mean continuous length calculation portion 83 calculates a meancontinuous length by averaging a plurality of continuous lengths thathave been detected by the continuous length detection portion 82.

The effect adjustment signal generation portion 84 outputs a specialimage effect adjustment signal DS according to the mean continuouslength, such the signal level (value) of the special image effectadjustment signal DS is smaller the longer the mean continuous length.For example, as shown in FIG. 34, it reduces the signal level (value) ofthe special image effect adjustment signal DS when the mean continuouslength that has been detected is within the range of the predeterminedvalue The to the predetermined value Thf. Here, the horizontal axis isthe mean continuous length, and the vertical axis is the output of thespecial image effect adjustment signal DS.

By providing threshold values in this way, it is possible for the effectadjustment signal generation portion 84 to create a special image effectadjustment signal DS for processing without weakening the visual effectin a case where this is below a threshold value The, which includesnormal images that are not special images. On the other hand, the effectadjustment signal generation portion 84 can create a special imageeffect adjustment signal DS in which the visual effect has beencompletely eliminated in a case where this is equal to or greater thanthe threshold Thf, which includes special images.

It should be noted that the range of the value of the special imageeffect adjustment signal DS has been set from 0.0 to 1.0, but it is alsopossible to adjust this to from 0.2 to 1.0, for instance, depending onthe strength of the visual processing. The visual processing device 1001is configured such that the effect of the visual processing becomesstronger the larger the value of the special image effect adjustmentsignal DS.

Thus, with the special image detection portion 80 of the second modifiedexample, it is possible to detect the magnitude of a special image thathas a bias of information from the image signal, and the special imagedetection portion 80 can be substituted for the special image detectionportion 2140.

A special image detection portion 90 according to a third modifiedexample is described next. In the third modified example, a statisticalbias of information is detected from the proportion of regions in whichthe gradation changes in the image of the image signal IS. Regions inwhich the gradation changes can be detected from edge components in theimage. Here, high frequency blocks that include high frequencycomponents are detected from a plurality of blocks that have beenobtained by partitioning as edge components in the image, and bydetecting the ratio of the number of high frequency blocks with respectto the total number of blocks that have been obtained by partitioning,the degree that an image is a special image is detected.

The case of the third modified example, in which the ratio of the numberof high frequency blocks is detected, is described using FIGS. 35, 36,and 37. FIG. 35 is a block diagram showing the configuration of thespecial image detection portion 90 of the third modified example, FIG.36 is an explanatory diagram for describing the block images of thethird modified example, and FIG. 37 is an explanatory diagram fordescribing the special image effect adjustment signal DS of the thirdmodified example.

As shown in FIG. 35, the special image detection portion 90 of the thirdmodified example is provided with a high-frequency block detectionportion 91 for detecting high-frequency blocks that includehigh-frequency components from an image signal IS that has beenpartitioned into a plurality of blocks, a high-frequency block densitydetection portion 92 for detecting the ratio of the number ofhigh-frequency blocks with respect to the total block number, and aneffect adjustment signal generation portion 93 for outputting an effectadjustment signal according to the ratio of the number of blocks thathas detected by the high-frequency block density detection portion 92.

The high-frequency block detection portion 91 can detect high-frequencycomponents in each encoded block in a case where the image signal thathas been input is a compressed image encoded by MPEG or JPEG, forexample. For example, it can extract high-frequency components bydetecting the AC coefficient of each encoded block.

The high-frequency block detection portion 91 determines that a block isa high-frequency block when a high-frequency component equal to orgreater than a predetermined value has been detected.

A case in which the special image 200 has been partitioned into aplurality of blocks as in FIG. 36, for example, and high-frequencycomponents are detected for each block, is described.

The high-frequency block detection portion 91 detects a high-frequencycomponent in the blocks 600 because they contain a edge of the imagepattern, and determines that these are “high-frequency blocks.” On theother hand, the high-frequency block detection portion 91 cannot detecta high-frequency component in the blocks 601 and 602 because they eachhave a substantially constant tone level (gradation value), anddetermines that each of these is “not a high-frequency block.”Hereinafter, it performs detection in the same manner for all of theblocks that have been obtained by partitioning.

The high-frequency block density detection portion 92 detects the ratioof the number of high-frequency blocks to the total number ofpartitioned blocks (hereinafter, this is called the “block density”).

Based on the block density, the effect adjustment signal generationportion 93 increases the value of the special image effect adjustmentsignal DS the higher the block density, and outputs the special imageeffect adjustment signal DS. For example, as shown in FIG. 37, theeffect adjustment signal generation portion 93 increases the value ofthe special image effect adjustment signal DS when the block densitythat has been detected is in the range of equal to or greater than apredetermined value Thg up to a predetermined value Thh. By providingthreshold values in this way, it is possible for the effect adjustmentsignal generation portion 93 to create a special image effect adjustmentsignal DS in which the visual effect has been completely eliminated ifthe block density is below the threshold Thg, which includes specialimages. On the other hand, the effect adjustment signal generationportion 93 can create a special image effect adjustment signal DS forprocessing without weakening the visual effect if the block density isgreater than a threshold value Thh, which includes normal images thatare not special images. Here, the horizontal axis is the block density,and the vertical axis is the output of the special image effectadjustment signal DS. It should be noted that the range of the value ofthe special image effect adjustment signal DS that is output has beenset from 0.0 to 1.0, but it is also possible to adjust this to from 0.2to 1.0, for instance, depending on the strength of the visualprocessing. The visual processing device 1001 is configured such thatthe effect of the visual processing becomes stronger the larger thevalue of the special image effect adjustment signal DS.

Thus, with the special image detection portion 90 of the third modifiedexample, it is possible to detect the degree of a special image that hasa bias in the image information from the image signal IS, and thespecial image detection portion 90 can be substituted for the specialimage detection portion 2140.

It should be noted that it is also possible for a special image having astatistical bias of information to be detected from a reduced image suchas a thumbnail image obtained by reducing the image signal, and then tooutput an effect adjustment signal based on that statistical bias ofinformation.

It is also possible to provide a reducing portion (not shown), forreducing the image signal, which is inserted between the input signal ISand the special image detection portion 2140, 700, 80, or 90, and thendetect special images having a statistical bias from the reduced imagethat is created by the reducing portion and output an effect adjustmentsignal MOD based on that statistical bias.

By using a reduced image, it is possible to detect flat regions that arenear edges while suppressing the effects of noise. In other words, thereis less of a noise component in a reduced image that has been created bya reduction method in which an image signal is first averaged and thenthinned out, and thus it is possible to detect a statistical bias ininformation while suppressing the effects of noise. Further, using areduced signal makes it possible to lower the number of pixels to bedetected, and this allows the number of calculations to be reduced.

Seventh Embodiment

With the visual processing device 1001 according to the sixth embodimentof the invention, a synthesized signal MUS that is synthesized alteringthe ratio of the image signal IS and the surrounding image information(unsharp signal) US according to an effect adjustment signal MOD isoutput, and the visual processing portion 30 outputs a processed signalOS that is obtained by visually processing the image signal according tothe synthesized signal MUS from the effect adjustment portion 1020, butwith a visual processing device 1002 according to the seventh embodimentof the invention, an effect adjustment portion 2021 outputs asynthesized output OUT that is obtained by synthesizing the processedsignal OS that has been visually processed and the image signal ISaccording to an effect adjustment signal. The visual processing device1002 according to the seventh embodiment of the invention is describedusing FIG. 38.

FIG. 38 is a block diagram that shows the configuration of the visualprocessing device 1002 according to the seventh embodiment of theinvention. Hereinafter, sections that are identical to those of thesixth embodiment are assigned the same reference numerals as before andwill not be described in detail.

In FIG. 38, the visual processing portion 30 outputs a processed signalOS based on the image signal IS and the output US of the spatialprocessing portion 10.

The effect adjustment portion 2021 interpolates the image signal IS andthe processed signal OS using the effect adjustment signal MOD in orderto differ the effect of the visual processing. For example, the outputOUT from the effect adjustment portion 2021 is calculated through aninterpolation calculation such as that of Eq. 9 below.OUT=OS×MOD+IS×(1.0−MOD)  Eq. 9

It should be noted that Eq. 9 can be modified as shown in Eq. 10.OUT=(OS−IS)×MOD+IS  Eq. 10

Thus, according to the visual processing device 1002 of the seventhembodiment of the invention, it is possible to output a synthesizedoutput OUT that is synthesized changing the proportion of the processedsignal OS and the image signal IS according to the effect adjustmentsignal MOD, and the effect of visual processing can be differed.

It should be noted that it is also possible to substitute the specialimage detection portion 2140 for the special image detection portion 700of the sixth embodiment of the invention. In this case as well, it ispossible to similarly detect special images and then create an effectadjustment signal MOD that corresponds to the bias of the imageinformation.

It should be noted that it is also possible to substitute the specialimage detection portion 2140 for the special image detection portion 80of the sixth embodiment of the invention. In this case as well, it ispossible to similarly detect special images and then create an effectadjustment signal MOD that corresponds to the bias of the imageinformation.

It should be noted that it is also possible to substitute the specialimage detection portion 2140 for the special image detection portion 90of the sixth embodiment of the invention. In this case as well, it ispossible to similarly detect special images and then create an effectadjustment signal MOD that corresponds to the bias of the imageinformation.

Eighth Embodiment

With the visual processing device 1001 of the sixth embodiment of theinvention, a synthesized signal MUS that is synthesized changing theproportion of the image signal IS and the surrounding image information(unsharp signal) US according to an effect adjustment signal MOD isoutput, and the visual processing portion 30 outputs a processed signalOS that is obtained by visually processing the image signal IS accordingto the synthesized signal MUS from the effect adjustment portion 1020,but with a visual processing device 1003 according to the eighthembodiment of the invention, an effect adjustment portion 2022 creates aprofile that is synthesized changing the proportion of the output ofeach of a plurality of profiles with different visual processing effectsin accordance with the effect adjustment signal MOD (hereinafter, thiswill be called a “synthesized profile”), and sets this in the LUT of thevisual processing portion 30. This embodiment is described using FIG.39.

FIG. 39 is a block diagram that shows the configuration of a visualprocessing device 1003 according to the eighth embodiment of theinvention. Hereinafter, sections that are identical to those of thesixth embodiment are assigned the same reference numerals as before andwill not be described in detail.

The effect adjustment portion 2022 synthesizes a third profile 6000 anda fourth profile 6001, which have different strengths of visualprocessing, through an interpolation computation based on the effectadjustment signal MOD in order to create a synthesized profile, and setsthis in the LUT of the visual processing portion 30. It should be notedthat it is also possible to create the synthesized profile through anextrapolation computation.

The visual processing portion 30 can perform visual processing withdifferent strengths of visual processing and different degrees of visualeffects using the synthesized profile that has been set in the LUT.

Thus, according to the visual processing device 1003 of the eighthembodiment of the invention, by synthesizing a plurality of profileswith different visual processing strengths and effects in accordancewith the effect adjustment signal MOD and then setting the synthesizedprofile in the LUT of the visual processing portion 30, it is possibleto differ the effect of visual processing.

It should be noted that it is also possible to substitute the specialimage detection portion 2140 for the special image detection portion 700of the sixth embodiment of the invention. In this case as well, it ispossible to similarly detect special images and then create an effectadjustment signal MOD that corresponds to the bias of information.

It should be noted that it is also possible to substitute the specialimage detection portion 2140 for the special image detection portion 80of the sixth embodiment of the invention. In this case as well, it ispossible to similarly detect special images and then create an effectadjustment signal MOD that corresponds to the bias of information.

It should be noted that it is also possible to substitute the specialimage detection portion 2140 for the special image detection portion 90of the sixth embodiment of the invention. In this case as well, it ispossible to similarly detect special images and then create an effectadjustment signal MOD that corresponds to the bias of information.

Ninth Embodiment

With the visual processing devices of the sixth embodiment of theinvention through the eighth embodiment of the invention, a toneconversion value based on two-dimensional tone conversioncharacteristics is output, but in the ninth embodiment of the invention,a gain-type visual processing system 1004 that performs tone conversionusing a gain output is described using FIGS. 40 and 41.

FIG. 40 is a block diagram that shows the configuration of a gain-typevisual processing system 1004 according to the ninth embodiment of theinvention, and FIG. 41 is an explanatory diagram for describing thetwo-dimensional gain characteristics. Hereinafter, sections that areidentical to those of the sixth embodiment are assigned the samereference numerals as before and will not be described in detail.

In FIG. 40, the gain-type visual processing system 1004 is provided witha gain-type visual processing device 4005 for outputting a gain signalGAIN that is obtained by visually processing the image signal IS, and amultiplier 4011 for multiplying the gain signal GAIN and the imagesignal

IS.

The gain-type visual processing device 4005 is provided with the visualprocessing device 1001 for outputting a processing signal OS obtained byvisually processing the image signal IS, and a divider 4012 for dividingthe processed signal OS by the image signal IS. Here, the visualprocessing device 1001 outputs a tone conversion value that is obtainedby visually processing the output of the image signal IS, and bydividing this tone conversion value by the image signal IS, it ispossible to achieve the gain-type visual processing device 4005.

The multiplier 4011 multiplies the image signal IS and the gain signalGAIN that is output by the gain-type visual processing device 4005, andoutputs a tone conversion value in which the output of the image signalIS has been visually processed.

It should be noted that it is also possible for the visual processingportion 30 to carry out processing by directly using a profile that hasthe two-dimensional gain characteristics shown in FIG. 41. Here, thevertical axis of FIG. 41 is the gain output GN, and the horizontal axisis the image signal IS. The two-dimensional gain characteristics shownin FIG. 41 are equivalent to those that are obtained by dividing theoutput of the profile of the two-dimensional tone conversioncharacteristics shown in FIG. 22 by the image signal IS. It is alsopossible to set a profile that has these two-dimensional gaincharacteristics in the LUT of the visual processing portion 30 of thevisual processing device 1001. By setting a profile of thesetwo-dimensional gain characteristics in the LUT of the visual processingportion 30 in advance in this way, the gain output GN and the gainsignal GAIN become equal and thus the divider 12 can be eliminated andit is still possible to achieve the gain-type visual processing device4005.

With the gain-type visual processing device 4005 in the gain-type visualprocessing system 1004 of the ninth embodiment of the invention, thereis little change in the processed signal that has been visuallyprocessed with respect to the change in the image signal IS that hasbeen input, and thus it is possible to reduce the number of bits of theinput signal and also to reduce the circuit scale. Additionally, if thevisual processing portion 30 is provided with a 2D LUT, then the memorycapacity can be reduced as well.

It should be noted that the visual processing device 1002 of the seventhembodiment of the invention can be substituted for the visual processingdevice 1001 of the sixth embodiment of the invention. The gain-typevisual processing device 4005 can be similarly achieved in this case aswell.

Likewise, the visual processing device 1003 of the eighth embodiment ofthe invention can be substituted for the visual processing device 1001of the sixth embodiment of the invention. The gain-type visualprocessing device 4005 can be similarly achieved in this case as well.

Thus, according to the sixth embodiment of the invention through theninth embodiment of the invention, the visual processing effect can bemaintained when a normal image that is not a special image has beeninput, and artifacts can be inhibited when a special image has beeninput.

Tenth Embodiment

The various functions such as the spatial processing function, effectadjustment function, visual processing function and the like in thevisual processing device or visual processing system according to thepresent invention explained in the aforementioned embodiments may becarried out by hardware using an integrated circuit, or by software thatoperates using a central processing unit (hereinafter, abbreviated as“CPU”), digital signal processor and the like. Alternatively, they maybe carried out by mixed processing using the hardware and software.

When the functions are carried out by the hardware, each function in theembodiments of the present invention may be achieved by a separateintegrated circuit, or a part or all of the functions may be achieved byone integrated circuit. The LSI may be referred to as an IC, a systemLSI, a super LSI or an ultra LSI in accordance with the degree ofintegration.

In addition, the integrating circuit may be achieved by an applicationspecific integrated circuit or a versatile processing unit. For example,it is possible to use an FPGA (Field Programmable Gate Array) that isprogrammable after the LSI is produced or a silicon figurable processorthat can restructure connection or setting of circuit cells in the LSI.

Furthermore, if another technique for integrating circuits rather thanthe LSI appears with the progress of semiconductor technology, then thattechnique may be utilized for integrating the functional blocks.Biotechnology has the potential for such technology.

Next, a case in which various functions are executed by software isdescribed using FIG. 46. FIG. 46 is a block diagram showing theconfiguration of a computer 4612 according to an embodiment of theinvention.

In FIG. 46, the computer 4612 is provided with a CPU 4600 that executesthe commands of various types of programs, a ROM 4601 storing programs,for example, a RAM 4602 holding temporarily stored data, an inputportion 4603 that inputs images, an output portion 4604 that outputsimages, and a memory portion 4605 that stores programs and various typesof data.

The computer 4612 also can be provided with a communication portion 4606for communicating with the outside, and a drive portion 4607 forsuitably connecting to information storage media.

The various functional portions send and receive control signals anddata, for example, via a bus 4610.

The CPU 4600 executes various functions according to programs stored onthe ROM 4601, programs stored on the memory portion 4605, and programsstored on the RAM 4602.

The ROM 4601 stores a visual processing program and characteristic data,for example.

The RAM 4602 temporarily stores data that are required for theprocessing of the various functions by the CPU 100.

The input portion 4603 inputs images. For example, it is possible forthe input portion 4603 to receive electromagnetic waves to obtainbroadcast image data, decode the broadcast image data and obtain videosignal. It is also possible to obtain digital images directly over awired connection.

The output portion 4604 outputs the images. For example, the outputportion 4604 outputs to a display device such as a liquid crystaldisplay device or a plasma display.

The memory portion 4605 is made of a magnetic memory and the like, andstores various programs or data.

The communication portion 4604 may be connected to the network 111 andthe like, and obtain the program via the network 111, or may install theobtained program in the memory portion 4605 as necessary. In this way,the computer 6 can download the program via the communication portion4606.The drive portion 4607 appropriately connects to an information storagemedium and obtains information stored therein. The information storagemedium may be, for example, the disk 4608 such as a magnetic disk,magneto optical disk, optical disk, or the memory card 4609 such as asemiconductor memory. In addition, the program having the variousfunctions, characteristic data and the like may be stored in the disk4608 or the memory card 4609 such as the semiconductor memory, and theinformation may be provided to the computer 4612.

A program can be incorporated into a computer in advance by dedicatedhardware, or it can be provided already incorporated into a ROM 4601 ora memory portion 4605.

The program can be adopted by devices that handle images, such asinformation processing devices, televisions, digital cameras, portabletelephones, and PDAs. The program can be installed in or connected to adevice that handles images, and executes the same visual processing asthe visual processing that is achieved by the visual processing devicesor visual processing systems described in above embodiments.

It should be noted that if the visual processing device is adopted in adisplay device, then it is also possible to switch the display mode whena special image is detected.

If the visual processing portion, for instance, of the visual processingdevices described in the above embodiments is constituted by a 2D LUT,then the data of the 2D LUT that is referenced are stored in a memorydevice such as a hard disk or a ROM, and are referenced as necessary. Itis also possible for the data of the 2D LUT to be provided from a devicefor providing the two-dimensional gain data (profile) for the 2D LUTthat is directly connected, or is indirectly connected via a network, tothe visual processing device.

A first visual processing method can be achieved for example by causinga computer 4612 to execute an effect adjustment step of performingprocessing for setting the effect of visual processing according to aneffect adjustment signal, a visual processing step of performing visualprocessing on an image signal that has been input, a target levelsetting step of setting a predetermined target level, and a spatialprocessing step of performing predetermined spatial processing on theimage signal and outputting a processed signal. The computer 4612 in theeffect adjustment step is made to output a synthesized signal that isobtained by synthesizing the processed signal and the predeterminedtarget level according to the effect adjustment signal, and in thevisual processing step is made to convert the tone level of the imagesignal based on the image signal and the synthesized signal that hasbeen synthesized.

A second visual processing method can be achieved for example by causinga computer 4612 to execute an effect adjustment step of performingprocessing for setting the effect of visual processing according to aneffect adjustment signal, a visual processing step of performing visualprocessing on an image signal that has been input, a surrounding imageinformation extraction step of extracting the surrounding imageinformation of the image signal that has been input, and an effectadjustment signal generation step of outputting an effect adjustmentsignal for setting the effect of visual processing. The computer 4612 inthe visual processing step is made to visually process the image signalbased on the image signal and the surrounding image information, and inthe effect adjustment step is made to set the effect of visualprocessing according to the effect adjustment signal.

A third visual processing method can be achieved for example by causinga computer 4612 to execute the second visual processing method, andalso, in the effect adjustment signal generation step, to detect flatregions that are adjacent to edge regions from the image signal andoutput an effect adjustment signal.

A fourth visual processing method can be achieved for example by causinga computer 4612 to execute an effect adjustment step of performingprocessing for setting the effect of visual processing according to aneffect adjustment signal, a visual processing step of performing visualprocessing on an image signal that has been input, a surrounding imageinformation extraction step of extracting the surrounding imageinformation of the image signal that has been input, and a special imagedetection step of detecting special images that have a statistical biasfrom the image signal and then outputting an effect adjustment signalbased on the degree of the statistical bias. Further, the computer 4612in the visual processing step is made to visually process the imagesignal based on the image signal and the surrounding image information,and in the effect adjustment step is made to set the effect of visualprocessing according to the effect adjustment signal.

Eleventh Embodiment

An example of the application of the visual processing device, as wellas a system using the same according to a second example of the presentinvention are described with reference to FIGS. 42 to 46.

FIG. 42 is a block diagram showing an overall structure of a contentproviding system ex100 that achieves a content delivering service. Anarea where a communication service is provided is divided into cells ofa desired size, and base stations ex107-ex110 that are fixed radiostations are provided in the cells.

This content providing system ex100 includes a computer ex111, apersonal digital assistant (PDA) ex112, a camera ex113, a cellular phoneex114, a cellular phone with camera ex115 and other equipment that areconnected to the Internet ex101 for example via an internet serviceprovider ex102, a telephone network ex104 and base stations ex107-ex110.

However, the content providing system ex100 can adopt any combinationfor connection without being limited to the combination shown in FIG.42. In addition, each of the devices can be connected directly to thetelephone network ex104 without the base stations ex107-ex110 that arefixed radio stations.

The camera ex113 is a device such as a digital video camera that canobtain a moving image. In addition, the cellular phone may be any typeof PDC (Personal Digital Communications) method, CDMA (Code DivisionMultiple Access) method, W-CDMA (Wideband-Code Division Multiple Access)method, or GSM (Global System for Mobile Communications) method, or acellular phone of PHS (Personal Handyphone System).

In addition, the streaming server ex103 is connected to the camera ex113via the base station ex109 and the telephone network ex104, so that livedelivery can be performed on the basis of coded data transmitted by auser of the camera ex113. The coding process of the obtained data may beperformed by the camera ex113 or by a server for transmitting data. Inaddition, the moving image data obtained by the camera ex116 may betransmitted to the streaming server ex103 via the computer ex111. Thecamera ex116 is a device that can take a still image like a digitalcamera and a moving image. In this case, coding of the moving image datamay be performed by the camera ex116 or by the computer ex111. Inaddition, the coding process may be performed by an LSI ex117 in thecomputer ex111 or the camera ex116. Note that it is possible toincorporate software for coding and decoding images into a storagemedium (a CD-ROM, a flexible disk, a hard disk or the like) that is arecording medium readable by the computer ex111. Furthermore, thecellular phone with camera ex115 may transmit the moving image data. Inthis case, the moving image data is coded by the LSI in the cellularphone ex115.

In this content providing system ex100, content (for example, a movingimage of a music concert) that the user is recording with the cameraex113 or the camera ex116 are coded as shown in the above-describedembodiments and transmitted to the streaming server ex103, while thestreaming server ex103 delivers a stream of the content data to a clientwho made a request. The client may be the computer ex111, the PDA ex112,the camera ex113, the cellular phone ex114 or the like that can decodethe coded data. Thus, in the content providing system ex100, the clientcan receive and reproduce the coded data. The system can achievepersonal broadcasting when the client receives, decodes and reproducesthe stream in real time.

To encode or decode the content, the visual processing devices may beused. For example, the computer ex111, the PDA ex112, the camera ex113,the cellular phone ex114 or the like may be provided with the visualprocessing devices, and execute the visual processing methods or thevisual processing programs.

In addition, the streaming server ex103 may provide the two-dimensionalgain data (profile) to the visual processing device via the Internetex101. Furthermore, there may be a plurality of streaming servers ex103,and each of them may provide different two-dimensional gain data.Further, the streaming sever ex103 may be for creating thetwo-dimensional gain data. When the visual processing device can thusobtain the two-dimensional gain data via the Internet ex101, the visualprocessing device does not have to store in advance the two-dimensionalgain data used for the visual processing, and the memory capacity of thevisual processing device can be reduced. In addition, because thetwo-dimensional gain data can be obtained from a plurality of serversconnected via the Internet ex101, different visual processings can beachieved.

An example regarding a cellular phone will now be described.

FIG. 43 shows the cellular phone ex115 that utilizes the visualprocessing device 1 of the present invention. The cellular phone ex115includes an antenna ex201 for transmitting and receiving radio waveswith the base station ex110, a camera portion ex203 such as a CCD camerathat can take a still image, a display portion ex202 such as a liquidcrystal display for displaying images obtained by the camera portionex203 or images received by the antenna ex201 after the image data aredecoded, a main body portion including a group of operating keys ex204,a sound output portion ex208 such as a speaker for producing sounds, asound input portion ex205 such as a microphone for receiving sounds, arecording medium ex207 for storing coded data or decoded data such asdata of taken moving images or still images, data of received e-mails,moving images or still images, and a slot portion ex206 that enables therecording medium ex207 to be attached to the cellular phone ex115. Therecording medium ex207 such as an SD card includes a plastic casehousing a flash memory element that is one type of EEPROM (ElectricallyErasable and Programmable Read Only Memory) nonvolatile memory that iselectronically rewritable and erasable.

Furthermore, the cellular phone ex115 will be described with referenceto FIG. 44. The cellular phone ex115 includes a main controller portionex311 for controlling each portion of the main body portion having thedisplay portion ex202 and the operating keys ex204, a power sourcecircuit portion ex310, an operational input controller portion ex304, animage coding portion ex312, a camera interface portion ex303, an LCD(Liquid Crystal Display) controller portion ex302, an image decodingportion ex309, a multiplex separation portion ex308, a recording andreproduction portion ex307, a modem circuit portion ex306 and a soundprocessing portion ex305, which are connected to each other via asynchronizing bus ex313.

When the user turns on a clear and power key, the power source circuitportion ex310 supplies power from a battery pack to each portion so thatthe digital cellular phone with camera ex115 is activated.

The cellular phone ex115 converts a sound signal collected by the soundinput portion ex205 during a sound communication mode into digital sounddata by the sound processing portion ex305 under control of the maincontroller portion ex311 that includes a CPU, a ROM and a RAM. Thedigital sound data are processed by the modem circuit portion ex306 as aspectrum spreading process and are processed by the transmission andreception circuit portion ex301 as a digital to analog conversionprocess and a frequency conversion process. After that, the data aretransmitted via the antenna ex201. In addition, the cellular phone ex115amplifies a signal that is received by the antenna ex201 during thesound communication mode and performs the frequency conversion processand an analog to digital conversion process on the data, which isprocessed by the modem circuit portion ex306 as a spectrum inversespreading process and is converted into a analog sound signal by thesound processing portion ex305. After that, the analog sound signal isdelivered by the sound output portion ex208.

Furthermore, when transmitting electronic mail during a datacommunication mode, text data of the electronic mail are entered byusing the operating keys ex204 of the main body portion and are given tothe main controller portion ex311 via the operational input controllerportion ex304. The main controller portion ex311 performs the spectrumspreading process on the text data by the modem circuit portion ex306and performs the digital to analog conversion process and the frequencyconversion process by the transmission and reception circuit portionex301. After that, the data are transmitted to the base station ex110via the antenna ex201.

When transmitting image data during the data communication mode, theimage data obtained by the camera portion ex203 are supplied to theimage coding portion ex312 via the camera interface portion ex303. Inaddition, if the image data are not transmitted, it is possible todisplay the image data obtained by the camera portion ex203 directly bythe display portion ex202 via the camera interface portion ex303 and anLCD controller portion ex302.

The image coding portion ex312 converts the image data supplied from thecamera portion ex203 into the coded image data by compressing and codingthe data, and the coded image data are supplied to the multiplexseparation portion ex308. In addition, the cellular phone ex115 collectssounds by the sound input portion ex205 while the camera portion ex203is taking the image, and the digital sound data is supplied from thesound processing portion ex305 to the multiplex separation portionex308.

The multiplex separation portion ex308 performs multiplexing of thecoded image data supplied from the image coding portion ex312 and thesound data supplied from the sound processing portion ex305 by apredetermined method. Multiplexed data obtained as a result areprocessed by the modem circuit portion ex306 as a spectrum spreadingprocess and are processed by the transmission and reception circuitportion ex301 as a digital to analog conversion process and a frequencyconversion process. After that, the data are transmitted via the antennaex201.

When receiving moving image file data linked to a web page during thedata communication mode, a signal received from the base station ex110via the antenna ex201 is processed by the modem circuit portion ex306 asa spectrum inverse spreading process. Multiplexed data obtained as aresult are supplied to the multiplex separation portion ex308.

In addition, in order to decode multiplexed data received via theantenna ex201, the multiplex separation portion ex308 separates a codedbit stream of image data in the multiplexed data from a coded bit streamof sound data. Then, the multiplex separation portion ex308 supplies thecoded image data to the image decoding portion ex309 via thesynchronizing bus ex313 and supplies the sound data to the soundprocessing portion ex305.

Next, the image decoding portion ex309 generates reproduction movingimage data by decoding the coded bit stream of the image data andsupplies the data to the display portion ex202 via the LCD controllerportion ex302. Thus, the moving image data included in a moving imagefile that is linked to a home page can be displayed. In this case, thesound processing portion ex305 converts the sound data into an analogsound signal, which is supplied to the sound output portion ex208. Thus,sound data included in the moving image file that is linked to a homepage can be reproduced.

Note that the image decoding portion ex309 may be provided with thevisual processing devices shown in the above-described embodiments.

Note that the present invention is not limited to the example of thesystem described above. Digital broadcasting by satellite or terrestrialsignals has been a recent topic of discussion. As shown in FIG. 45, thevisual processing devices of the present invention can be incorporatedinto the digital broadcasting system, too.

More specifically, in a broadcast station ex409, a coded bit stream ofimage information is sent to a communication or a broadcasting satelliteex410 via a radio wave. The broadcasting satellite ex410 that receivedthe coded bit stream of image information sends radio waves forbroadcasting. These radio waves are received by an antenna ex406 of ahouse equipped with a satellite broadcasting reception facility, and adevice such as a television set (a receiver) ex401 or a set top box(STB) ex407 decodes the coded bit stream and reproduces the same. Inaddition, the television set (the receiver) ex401 or the set top box(STB) ex407 may be provided with the visual processing device shown inthe above-described embodiments, use the visual processing method shownin the above-described embodiments or execute the visual processingprogram shown in the above-described embodiments. In addition, areproduction device ex403 for reading and decoding a coded bit streamthat is recorded on a storage medium ex402 such as a CD or a DVD that isa recording medium may be equipped with the visual processing devices,the visual processing methods, and the visual processing programs shownin the above-described embodiments. In this case, the reproduced imagesignal and text track are displayed on a monitor ex404. In addition, itis possible to mount the visual processing devices, the visualprocessing methods, and the visual processing programs shown in theabove-described embodiments, in a set top box ex407 that is connected toa cable ex405 for a cable television or the antenna ex406 for asatellite or surface wave broadcasting, so that the image can bereproduced on a monitor ex408 of the television set. In this case, it ispossible to incorporate the visual processing devices shown in theabove-described embodiments not into the set top box but into thetelevision set. In addition, it is possible that a car ex412 equippedwith an antenna ex411 receives a signal from the broadcasting satelliteex410 or the base station ex107 and reproduces the moving image on adisplay of a navigation system ex413 in the car ex412.

Furthermore, it is possible to encode the image signal and record theencoded image signal in a recording medium. As a specific example, thereis a recorder ex420 such as a DVD recorder for recording image signalson a DVD disk ex421 or a disk recorder for recording image signals on ahard disk. Furthermore, it is possible to record on an SD card ex422. Inaddition, in case that the recorder ex420 includes the visual processingdevices of the present invention, it is possible to reproduce imagesignals recorded on a DVD disk ex421 or a SD card ex422 via the imagesignal processing device, so as to display on the monitor ex408.

Note that in the structure of the navigation system ex413 shown in FIG.44, the camera portion ex203, the camera interface portion ex303 and theimage coding portion ex312 can be omitted. This can be also applied tothe computer ex111 and the television set (the receiver) ex401.

In addition, the terminal device such as the cellular phone ex114 mayinclude three types of assemblies. A first type is a transmission andreception terminal having both the coder and the decoder, a second typeis a transmission terminal having only a coder and a third type is areception terminal having only a decoder.

It should be noted that the specific configuration of the invention isnot limited to the foregoing embodiments, and various changes andmodifications are possible in a range that does not depart from the gistof the invention.

INDUSTRIAL APPLICABILITY

With the visual processing device, display device, visual processingmethod, program, and integrated circuit according to the invention, itis possible to inhibit artifacts even if an image that has sharp edgeregions or a special image has been input, and with a simpleconfiguration it is possible to change the strength of the visualprocessing of the image in real-time, and thus these can be employed indisplay devices such as color television receivers, portable devices,information processing devices, digital still cameras, and gamemachines, and in output devices such as projectors and printers.

1. A visual processing device comprising: a spatial processing portionoperable to output a processed signal generated by performingpredetermined spatial processing on an input image signal based onpixels surrounding a target pixel of the input image signal, so as toreduce spatial high frequency components of the pixels surrounding thetarget pixel of the input image signal; a target level setting portionoperable to set a predetermined target level; an effect adjustmentportion operable to output a correction signal obtained by combining thepredetermined target level and the processed signal, the predeterminedtarget level and the processed signal being combined according to aneffect adjustment signal for setting an effect of visual processing; anda visual processing portion operable to (i) receive the correctionsignal and the input image signal, (ii) perform the visual processing onthe input image signal, and (iii) output an output signal as a result ofthe visual processing, the visual processing being performed using theinput image signal and the correction signal output by the effectadjustment portion, wherein the visual processing portion performs thevisual processing using tone conversion characteristics, such that, whena value of the input image signal is a predetermined value, acorresponding value of the output signal monotonically decreases withrespect to a value of the correction signal.
 2. A visual processingdevice comprising: a spatial processing portion operable to output aprocessed signal generated by performing predetermined spatialprocessing on an input image signal based on pixels surrounding a targetpixel of the input image signal, so as to reduce spatial high frequencycomponents of the pixels surrounding the target pixel of the input imagesignal; an effect adjustment portion operable to output a correctionsignal obtained by combining the input image signal and the processedsignal, the input image signal and the processed signal being combinedaccording to an effect adjustment signal for setting an effect of visualprocessing; and a visual processing portion operable to (i) receive thecorrection signal and the input image signal, (ii) perform the visualprocessing on the input image signal, and (iii) output an output signalas a result of the visual processing, the visual processing beingperformed using the input image signal and the correction signal outputby the effect adjustment portion, wherein the visual processing portionperforms the visual processing using tone conversion characteristics,such that, when a value of the input image signal is a predeterminedvalue, a corresponding value of the output signal monotonicallydecreases with respect to a value of the correction signal.
 3. Thevisual processing device according to claim 1, wherein the visualprocessing portion includes: a gain signal outputting portion operableto output a gain signal according to the correction signal and the inputimage signal; and a correction portion operable to correct the inputimage signal based on the gain signal and output the corrected inputimage signal as the output signal.
 4. The visual processing deviceaccording to claim 1, further comprising an effect adjustment signalgenerating portion operable to output the effect adjustment signal,wherein the effect adjustment signal generating portion extracts animage characteristic amount from the input image signal and generatesthe effect adjustment signal according to the extracted imagecharacteristic amount.
 5. The visual processing device according toclaim 4, wherein the effect adjustment signal generating portionextracts an image gradation level from the input image signal andgenerates the effect adjustment signal according to the extracted imagegradation level.
 6. The visual processing device according to claim 4,wherein the effect adjustment signal generating portion extracts colorinformation from the input image signal and generates the effectadjustment signal according to the extracted color information.
 7. Animage display device comprising: the visual processing device accordingto claim 1; and a display portion operable to display the output signaloutput from the visual processing device.
 8. A television devicecomprising: a reception portion operable to receive a video signal; adecoding portion operable to decode the received video signal and outputan image signal; a spatial processing portion operable to output aprocessed signal generated by performing predetermined spatialprocessing on the image signal based on pixels surrounding a targetpixel of the image signal, so as to reduce spatial high frequencycomponents of the pixels surrounding the target pixel of the input imagesignal; a target level setting portion operable to set a predeterminedtarget level; an effect adjustment portion operable to output acorrection signal obtained by combining the predetermined target leveland the processed signal, the predetermined target level and theprocessed signal being combined according to an effect adjustment signalfor setting an effect of visual processing; a visual processing portionoperable to (i) receive the correction signal and the image signal, (ii)perform the visual processing on the image signal, and (iii) output anoutput signal as a result of the visual processing, the visualprocessing being performed using the image signal and the correctionsignal output by the effect adjustment portion; and a display portionoperable to display the output signal, wherein the visual processingportion performs the visual processing using tone conversioncharacteristics, such that, when a value of the image signal is apredetermined value, a corresponding value of the output signalmonotonically decreases with respect to a value of the correctionsignal.
 9. A portable information terminal device comprising: areception portion operable to receive a video signal; a decoding portionoperable to decode the received video signal and output an image signal;a spatial processing portion operable to output a processed signalgenerated by performing predetermined spatial processing on the imagesignal based on pixels surrounding a target pixel of the image signal,so as to reduce spatial high frequency components of the pixelssurrounding the target pixel of the input image signal; a target levelsetting portion operable to set a predetermined target level; an effectadjustment portion operable to output a correction signal obtained bycombining the predetermined target level and the processed signal, thepredetermined target level and the processed signal being combinedaccording to an effect adjustment signal for setting an effect of visualprocessing; a visual processing portion operable to (i) receive thecorrection signal and the image signal, (ii) perform the visualprocessing on the image signal, and (iii) output an output signal as aresult of the visual processing, the visual processing being performedusing the image signal and the correction signal output by the effectadjustment portion; and a display portion operable to display the outputsignal, wherein the visual processing portion performs the visualprocessing using tone conversion characteristics, such that, when avalue of the image signal is a predetermined value, a correspondingvalue of the output signal monotonically decreases with respect to avalue of the correction signal.
 10. A camera comprising: an imagingportion operable to capture an image and generate an image signal; aspatial processing portion operable to output a processed signalgenerated by performing predetermined spatial processing on the imagesignal based on pixels surrounding a target pixel of the image signaland output a processed signal, so as to reduce spatial high frequencycomponents of the pixels surrounding the target pixel of the input imagesignal; a target level setting portion operable to set a predeterminedtarget level; an effect adjustment portion operable to output acorrection signal obtained by combining the predetermined target leveland the processed signal, the predetermined target level and theprocessed signal being combined according to an effect adjustment signalfor setting an effect of visual processing; and a visual processingportion operable to (i) receive the correction signal and the imagesignal, (ii) perform the visual processing on the image signal, and(iii) output an output signal as a result of the visual processing, thevisual processing being performed using the image signal and thecorrection signal output by the effect adjustment portion, wherein thevisual processing portion performs the visual processing using toneconversion characteristics, such that, when a value of the image signalis a predetermined value, a corresponding value of the outputsignal-monotonically decreases with respect to a value of the correctionsignal.
 11. A visual processing method comprising: a spatial processingstep of outputting a processed signal generated by performingpredetermined spatial processing on an input image signal based onpixels surrounding a target pixel of the input image signal andoutputting a processed signal, so as to reduce spatial high frequencycomponents of the pixels surrounding the target pixel of the input imagesignal; a target level setting step of setting a predetermined targetlevel; an effect adjustment step of outputting a correction signalobtained by combining the predetermined target level and the processedsignal, the predetermined target level and the processed signal beingcombined according to an effect adjustment signal for setting an effectof visual processing; and a visual processing step of (i) receiving thecorrection signal and the input image signal, (ii) performing the visualprocessing on the input image signal, and (iii) outputting an outputsignal as a result of the visual processing, the visual processing beingperformed using the input image signal and the correction signal outputby the effect adjustment step, wherein the visual processing stepincludes performing the visual processing using tone conversioncharacteristics, such that, when value of the input image signal is apredetermined value, a corresponding value of the output signalmonotonically decreases with respect to a value of the correctionsignal.
 12. An integrated circuit executing the visual processing methodaccording to claim
 11. 13. A non-transitory computer-readable storagemedium storing an image processing program for causing a computer toexecute the visual processing method according to claim
 11. 14. An imagedisplay device by comprising: the visual processing device according toclaim 2; and a display portion operable to display the output signaloutput from the visual processing device.
 15. A television devicecomprising: a reception portion operable to receive a video signal; adecoding portion operable to decode the received video signal and outputan image signal; a spatial processing portion operable to output aprocessed signal generated by performing predetermined spatialprocessing on the image signal based on using pixels surrounding atarget pixel of the image signal, so as to reduce spatial high frequencycomponents of the pixels surrounding the target pixel of the input imagesignal; an effect adjustment portion operable to output a correctionsignal obtained by combining the image signal and the processed signal,the image signal and the processed signal being combined according to aneffect adjustment signal for setting an effect of visual processing; avisual processing portion operable to (i) receive the correction signaland the image signal, (ii) perform the visual processing on the imagesignal, and (iii) output an output signal as a result of the visualprocessing, the visual processing being performed using the image signaland the correction signal output by the effect adjustment portion; and adisplay portion operable to display the output signal, wherein thevisual processing portion performs the visual processing using toneconversion characteristics, such that, when a value of the image signalis a predetermined value, a corresponding value of the output signalmonotonically decreases with respect to a value of the correctionsignal.
 16. A portable information terminal device comprising: areception portion operable to receive a video signal; a decoding portionoperable to decode the received video signal and output an image signal;a spatial processing portion operable to output a processed signalgenerated by performing predetermined spatial processing on the imagesignal based on using pixels surrounding a target pixel of the imagesignal and output a processed signal, so as to reduce spatial highfrequency components of the pixels surrounding the target pixel of theinput image signal; an effect adjustment portion operable to output acorrection signal obtained by combining the image signal and theprocessed signal, the image signal and the processed signal beingcombined according to an effect adjustment signal for setting an effectof visual processing; a visual processing portion operable to (i)receive the correction signal and the image signal, (ii) perform thevisual processing on the image signal, and (iii) output an output signalas a result of the visual processing, the visual processing beingperformed using the image signal and the correction signal output by theeffect adjustment portion; and a display portion operable to display theoutput signal, wherein the visual processing portion performs the visualprocessing using tone conversion characteristics, such that, when avalue of the image signal is a predetermined value, a correspondingvalue of the output signal monotonically decreases with respect to avalue of the correction signal.
 17. A camera comprising: an imagingportion operable to capture an image and generate an image signal; aspatial processing portion operable to output a processed signalgenerated by performing predetermined spatial processing on the imagesignal based on pixels surrounding a target pixel of the image signal,so as to reduce spatial high frequency components of the pixelssurrounding the target pixel of the input image signal; an effectadjustment portion operable to output a correction signal obtained bycombining the image signal and the processed signal, the image signaland the processed signal being combined according to an effectadjustment signal for setting an effect of visual processing; and avisual processing portion operable to (i) receive the correction signaland the image signal, (ii) perform the visual processing on the imagesignal, and (iii) output an output signal as a result of the visualprocessing, the visual processing being performed using the image signaland the correction signal output by the effect adjustment portion,wherein the visual processing portion performs the visual processingusing tone conversion characteristics, such that, when a value of theimage signal is a predetermined value, a corresponding value of theoutput signal monotonically decreases with respect to a value of thecorrection signal.
 18. A visual processing method comprising: a spatialprocessing step of outputting a processed signal generated by performingpredetermined spatial processing on an input image signal based onpixels surrounding a target pixel of the input image signal, so as toreduce spatial high frequency components of the pixels surrounding thetarget pixel of the input image signal; an effect adjustment step ofoutputting a correction signal obtained by combining the input imagesignal and the processed signal, the input image signal and theprocessed signal being combined according to an effect adjustment signalfor setting an effect of visual processing; and a visual processing stepof (i) receiving the correction signal and the input image signal, (ii)performing the visual processing on the input image signal, and (iii)outputting an output signal as a result of the visual processing, thevisual processing being performed using the input image signal and thecorrection signal output by the effect adjustment step, wherein thevisual processing step includes performing the visual processing usingtone conversion characteristics, such that, when a value of the inputimage signal is a predetermined value, a corresponding value of theoutput signal monotonically decreases with respect to a value of thecorrection signal.
 19. An integrated circuit executing the visualprocessing method according to claim
 18. 20. A non-transitorycomputer-readable storage medium storing an image processing program forcausing a computer to execute the visual processing method according toclaim 19.